Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

PrivateCloudComputeLanguageModel — session.respond hangs for minutes then throws FoundationModels.LanguageModelError -1 wrapping GenerativeFunctionsFoundation.GenerativeError 5040000 (macOS app, not a simulator)
I'm trying to add Private Cloud Compute support to a macOS app and can't get session.respond to succeed. Every call — even a trivial one-line prompt — hangs for one to several minutes ("Thinking" in my UI) and then fails. My setup: macOS 27.2 beta on an M2 MBP Xcode 27.2 The Private Cloud Compute capability is added in Signing & Capabilities, and com.apple.developer.private-cloud-compute is present in the built app's entitlements Removing the entitlement produces the expected error at construction ("Missing entitlement: com.appledeveloper.private-cloud-compute"), confirming it's genuinely being read PrivateCloudComputeLanguageModel().availability reports .available Apple Intelligence is enabled, I'm signed into iCloud with an eligible account/region, and system language is English The failure reproduces with zero tools registered on the session, and with a single isolated request. Relevant code: swift let model = PrivateCloudComputeLanguageModel() let session = LanguageModelSession(model: model, instructions: "You are a helpful assistant.") let result = try await session.respond(to: "What is 17+10?") The error, in full, after several minutes: Error Domain=FoundationModels.LanguageModelError Code=-1 "The operation couldn't be completed. (FoundationModels.LanguageModelError error -1.)" UserInfo={NSLocalizedDescription=The operation couldn't be completed. (FoundationModels.LanguageModelError error -1.), NSMultipleUnderlyingErrorsKey=( "Error Domain=FoundationModels.LanguageModelError Code=-1 "(null)" UserInfo={NSMultipleUnderlyingErrorsKey=(\n "Error Domain=com.apple.GenerativeFunctionsFoundation.GenerativeError Code=5040000 \"(null)\" UserInfo={NSMultipleUnderlyingErrorsKey=(\n)}"\n)}" )} Questions: Is this a known issue with Private Cloud Compute for macOS apps at this point in the beta? Does GenerativeFunctionsFoundation.GenerativeError code 5040000 mean anything specific — a connectivity failure, a capacity/availability issue, something else? Is there a way to get more diagnostic detail than this generic top-level error — a specific log subsystem to check in Console, for instance? Any guidance appreciated — happy to provide a full sysdiagnose or additional repro detail if useful.
1
0
517
14h
Best Practices for Building AI-Powered Features in iOS Applications
I am exploring different approaches for integrating AI capabilities into modern iOS applications and would like to learn from developers who have built AI-powered experiences. Some areas I am interested in: Best architecture patterns for AI-powered iOS apps Handling API communication securely Managing latency and offline scenarios Protecting user data when working with AI services Designing a reliable user experience around AI-generated responses For developers who have implemented AI features in production apps: What frameworks, architectures, or patterns have worked well for you? Are there any common mistakes you would recommend avoiding when building AI-powered applications on Apple platforms?
0
0
425
1d
iPhone 16 Pro failing to install new Siri Beta
I am currently on Apple's Dev Beta V4 for iOS 27. The first version I installed was the Dev Beta V2, I am desperate to try out the new Siri AI Beta, but it's just not installing for me. I have the ability to "turn siri off" then "on again" and find I get the 2024 Apple Intelligence version fine. But if I choose to try out the new AI Beta, I'm left with "Adding support for Siri is in progress. Siri will be unavailable until the update is complete." It's been in that state for over 48 hours in Beta 4 and I'm left with the OLD OLD Siri globe from pre-Apple intelligence. Am I being too keen and just not leaving it long enough? Or is there a genuine issue at Apple's end, in regard to getting the new Siri to actually fully install?
42
5
12k
1d
macOS 27.0 (26A428): Core ML multifunction ML Program is recognized by MLModelAsset but fails to load
Hello, We are seeing what appears to be a regression in the Core ML multifunction ML Program loading path on macOS 27.0. A compiled multifunction ML Program is correctly recognized by MLModelAsset and MLModelStructure, but loading either named function through MLModel fails with an error claiming that the model is not an ML Program. Environment macOS 27.0 Build: 26A428 Apple silicon Mac BABANE 1.0.4, build 16 Application built with the macOS 26.5 SDK Reproduces both inside and outside App Sandbox Approximately 98 GiB of disk space is available Public reproduction BABANE is available from the Mac App Store: BABANE on the App Store Apple engineers can reproduce the issue without receiving a separate model archive: Install BABANE from the App Store on macOS 27.0. Download either available translation model in the app. The model is delivered using Apple-Hosted Background Assets. Trigger model loading by starting a translation. Core ML fails while loading the first named function. The downloadable models are approximately 1.9 GB, so the App Store build is the most practical complete reproduction environment. Model structure The model is a specification-version-9 ML Program containing two functions: infer prefill Core ML correctly recognizes both functions: let asset = try MLModelAsset(url: compiledModelURL) let functionNames = try await asset.functionNames print(functionNames) Output: ["infer", "prefill"] MLModelStructure also returns a .program structure containing both functions. Loading code import CoreML func loadModel( at url: URL, functionName: String? ) throws -> MLModel { let configuration = MLModelConfiguration() configuration.computeUnits = .cpuAndNeuralEngine configuration.functionName = functionName return try MLModel( contentsOf: url, configuration: configuration ) } Loading either function: try loadModel(at: compiledModelURL, functionName: "infer") or: try loadModel(at: compiledModelURL, functionName: "prefill") fails with: `MLModelConfiguration`'s `.functionName` property must be `nil` unless the model type is ML Program. This contradicts the results returned by MLModelAsset and MLModelStructure. Setting functionName to nil is not a workaround. It fails with: This MLModel doesn't support the multi-function description syntax. Unified logging Immediately before the public Core ML error, unified logging reports: E5RT encountered an STL exception. E5RT: <private> (11) Core ML then returns the misleading functionName error. Tests performed We tested: functionName = "infer" functionName = "prefill" functionName = nil .cpuOnly .cpuAndGPU .cpuAndNeuralEngine .all App Sandbox application Non-sandboxed command-line executable Existing .mlmodelc A newly compiled .mlmodelc produced on macOS 27 All named-function combinations fail in the same way. The failure is independent of compute-unit selection and App Sandbox. The source package recompiles successfully on macOS 27, but the newly compiled model still fails to load. As an additional control: A system-provided multifunction ML Program exhibits the same loading failure on this installation. A single-function Core ML model loads successfully. This appears specific to the multifunction model loading path. Documentation The current Core ML documentation still describes MLModelAsset.functionNames as the way to discover functions and MLModelConfiguration.functionName as the way to select one: MLModelConfiguration.functionName MLModelAsset.functionNames We could not find any macOS 27 documentation or release-note entry stating that this behavior changed, that named functions now require a different loading API, or that a new entitlement is required. We found some potentially related reports: Core ML loading crash on macOS 27.0 build 26A428 Historical multifunction model loading crash Core ML/E5RT AOT loading regression with an Apple DTS response None of these reports documents the exact functionName failure described here. Expected behavior A model recognized as a multifunction ML Program should load when MLModelConfiguration.functionName is set to one of the names returned by MLModelAsset.functionNames. Actual behavior MLModel rejects the named function and incorrectly reports that the model is not an ML Program. Questions Is this a known macOS 27.0 regression in the Core ML multifunction loading path? Does MLModelConfiguration.functionName still accept names returned by MLModelAsset.functionNames on macOS 27? Is there a new required loading API, deployment target, SDK, entitlement, or model-packaging rule? Is there a supported workaround other than exporting each function as a separate model? Which diagnostics should we attach to a Feedback Assistant report besides the reproducer, unified logs, sysdiagnose, and exact OS/Xcode builds? Thank you.
0
0
231
1d
Error in Xcode console
Lately I am getting this error. GenerativeModelsAvailability.Parameters: Initialized with invalid language code: en-GB. Expected to receive two-letter ISO 639 code. e.g. 'zh' or 'en'. Falling back to: en Does anyone know what this is and how it can be resolved. The error does not crash the app
5
2
2.0k
1d
FoundationModels guided generation: empty token masks and severe slowdowns on macOS 27 betas 5, 6 and 7
Has anyone else hit this? We have a Mac app that uses FoundationModels with @Generable types for structured output. Starting with macOS 27 beta 5 every guided generation request began logging tokenizer errors and long structured requests slowed from seconds to minutes. Beta 6 and beta 7 both still have it. Filed as FB24310823 on August 11 with a full sysdiagnose and log captures, and we have appended evidence from each beta since. The signature is easy to check. Stream the log while your app generates: log stream --predicate 'subsystem == "com.apple.tokengenerationcore"' --style compact On an affected machine the inference service (TGOnDeviceInferenceProviderService, category guided) prints these two lines in matched pairs, thousands of times: Generated an empty mask at recognizer index N allowedTokenIDs is empty. Something is likely wrong with the tokenizer What we measured on beta 7 today: 9,008 of those pairs in about five and a half minutes of scanning. The errors start about one second into the first request after a fresh app launch, so it needs no warmup. Requests that normally finish in 4 to 12 seconds take 77 to 170 seconds or longer. On beta 5 we measured decode at roughly 0.3 tokens per second on the worst requests. Short requests still finish at normal speed but they emit the same errors while they run, and the quality of the structured content they return is degraded. On betas 5 and 6 we also saw repeated asset release errors for instruct_300m.tokenizer and the instruct_3b tokenizer saying the asset is not marked as in use. For what it is worth, a build that ran clean on beta 4 shows the same behavior on beta 5 and later with no app changes, and the same @Generable schema drives both the fast and the slow requests. But we know that does not rule out something on our side, and we would honestly be happy to learn this is our own bug since that would mean we can fix it. So two questions. Is anyone else seeing this since beta 5? And if you spot something we might be doing wrong on our end, sessions we should be recreating, schema patterns that stress the constrained decoder, anything at all, we would really appreciate the feedback. If it does turn out you are hitting the same thing, a Feedback referencing FB24310823 would help a lot. Thanks!
20
1
3.5k
2d
MLModelAsset(specification:blobMapping:) with mlprogram model: correct predictions but drastically slower inference than compiled .mlmodelc path
I'm distributing an encrypted .mlpackage to my app and want to load it entirely in memory without ever writing decrypted weights to disk. I tried MLModelAsset(specification:blobMapping:) as the path to achieve this, but ran into a significant inference performance gap compared to the compiled code path. What I'm trying to do The encrypted .enc file is a serialized FileWrapper of the full .mlpackage, sealed with AES-GCM. At runtime I decrypt it in memory, deserialize the FileWrapper, extract the spec and weight blob, and load via MLModelAsset: static func loadEncryptedPackage(url: URL, configuration: MLModelConfiguration) async throws -> MLModel { // AES-GCM decryption → decryptedData (full serialized .mlpackage) guard let wrapper = FileWrapper(serializedRepresentation: decryptedData) else { throw ... } guard let (specWrapper, specParent) = findSpecWrapper(in: wrapper), let spec = specWrapper.regularFileContents else { throw ... } var blobs: [URL: Data] = [:] collectBlobs(in: specParent, relativePath: "", excluding: specWrapper, into: &blobs) // keys built as URL(fileURLWithPath: rel), e.g. "weights/weight.bin" let asset = try MLModelAsset(specification: spec, blobMapping: blobs) let model = try await MLModel.load(asset: asset, configuration: configuration) // See observation #3 below — must retain these for the model's lifetime objc_setAssociatedObject(model, &retentionKey, Retainer(spec: spec, blobs: blobs), .OBJC_ASSOCIATION_RETAIN) return model } What I observed Predictions are accurate. The blobs are found, weights are applied, and the model produces correct results. Inference is drastically slower than the compiled code path. The same model loaded via MLModel.compileModel(at:) + MLModel.load(contentsOf:) runs inference much faster on the same device with the same MLModelConfiguration (computeUnits = .all). With MLModelAsset the slowdown is consistent across every prediction call, not just the first one. The spec and blob Data objects must stay alive for the model's lifetime. Without retaining them via objc_setAssociatedObject, inference produces NaN outputs or crashes. This suggests Core ML holds a reference back into those Data buffers beyond the load() call, rather than copying them into its own memory during loading. Using the exact blob URI from the spec as the blobMapping key triggers a compilation error. The spec (inspected via strings on the .mlmodel protobuf) stores blob references as @model_path/weights/weight.bin. When I key the blobMapping with URL(string: "@model_path/weights/weight.bin"), MLModel.load(asset:) throws: compiler error: Encountered an error while compiling a model: validator error: The in-memory ML Program must not have a blob file reference but found a reference to mem://weights/weight.bin. With other key formats (e.g. URL(fileURLWithPath: "weights/weight.bin")), this error does not appear — the model loads and predictions are accurate, but inference is slow as in observation #2. The working alternative (which I want to avoid) Decrypting to a temporary directory, calling MLModel.compileModel(at:), loading from the compiled .mlmodelc, then deleting the temp files produces fast inference. Same model, same device, same configuration. The only difference is the compilation step — and the fact that decrypted weights touch disk, which I want to avoid for security reasons. Questions Is MLModelAsset(specification:blobMapping:) expected to produce inference performance equivalent to loading from a compiled .mlmodelc? If not, is the performance gap fundamental to the API or something that can be addressed? Is there any supported way to load an mlprogram model with external weight blobs entirely in memory and achieve inference performance comparable to the compiled code path — i.e. without writing decrypted model data to disk at any point? The validator error "in-memory ML Program must not have a blob file reference" is a hard block when Core ML successfully resolves the blobs and attempts mlprogram compilation. Is this an intended constraint, and does it mean MLModelAsset(specification:blobMapping:) is not the right API for this use case?
2
1
663
2d
Crakling with iOS 27
I developed a conversational agent using Gradium and it worked fine using the iPhone's built-in speaker on iOS 18.7 and iOS 26.6. I developed similar conversational agent using the Gemini stack and it also worked fine using the iPhone's built-in speaker on iOS 18.7 and iOS 26.6. Now on iOS 27.0, we hear crackling in the transcripts and responses, with both stacks.
1
0
43
2d
modelWithContentsOfURL crashed (Apple M4 Pro)
Process: MagiMir2 [5177] Path: /Applications/MagiMir2.app/Contents/MacOS/MagiMir2 Identifier: com.magic.magimir Version: 2.0.2 (2.0.2) Code Type: ARM-64 (Native) Role: Foreground Parent Process: zsh [3124] Coalition: com.apple.Terminal [1945] Responsible Process: Terminal [2656] User ID: 501 Date/Time: 2026-09-16 01:16:23.1769 +0700 Launch Time: 2026-09-16 01:16:21.1311 +0700 Hardware Model: Mac16,7 OS Version: macOS 27.0 (26A428) Release Type: User Thread 19 Crashed:: Dispatch queue: com.apple.coreml.MLModelAssetResourceFactory.modelLoadQueue 0 libsystem_platform.dylib 0x18e1bba50 __bzero + 32 1 libc++.1.dylib 0x18e0beca8 std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>::append(unsigned long, char) + 156 2 CoreML 0x199be7d50 operator>>(IArchive&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>&) + 64 3 CoreML 0x199b57fd4 -[MLLoaderEvent extractAndSetModelDetailsFromArchive:] + 336 4 CoreML 0x199b27c88 +[MLLoader _loadModelFromArchive:configuration:loaderEvent:useUpdatableModelLoaders:error:] + 96 5 CoreML 0x199b25938 +[MLLoader _loadModelFromAssetAtURL:configuration:loaderEvent:error:] + 280 6 CoreML 0x199b256c0 +[MLLoader loadModelFromAssetAtURL:configuration:error:] + 112 7 CoreML 0x199a6f18c -[MLModelAssetResourceFactoryOnDiskImpl modelWithConfiguration:error:] + 120 8 CoreML 0x199bd29b4 __60-[MLModelAssetResourceFactory modelWithConfiguration:error:]_block_invoke + 72 9 libdispatch.dylib 0x18e00a5a0 _dispatch_client_callout + 16 10 libdispatch.dylib 0x18e000490 _dispatch_lane_barrier_sync_invoke_and_complete + 56 11 CoreML 0x199bd287c -[MLModelAssetResourceFactory modelWithConfiguration:error:] + 284 12 CoreML 0x199be8ae8 -[MLModelAssetModelVendor modelWithConfiguration:error:] + 156 13 CoreML 0x199b04a78 -[MLModelAsset modelWithConfiguration:error:] + 116 14 CoreML 0x199b89e94 +[MLModel modelWithContentsOfURL:configuration:error:] + 176 15 MagpieMobile 0x1026dd974 0x10213c000 + 5904756 16 MagpieMobile 0x102710d28 0x10213c000 + 6114600 17 magic_flutter_plugin_magpie 0x100e5c40c closure #1 in MagicFlutterPluginMagpiePlugin.preloadModels(result:) + 192 18 magic_flutter_plugin_magpie 0x100e56134 thunk for @escaping @callee_guaranteed () -> () + 28 19 libdispatch.dylib 0x18dff0a34 _dispatch_call_block_and_release + 32 20 libdispatch.dylib 0x18e00a5a0 _dispatch_client_callout + 16 21 libdispatch.dylib 0x18dff8fc0 _dispatch_lane_serial_drain + 744 22 libdispatch.dylib 0x18dff9abc _dispatch_lane_invoke + 392 23 libdispatch.dylib 0x18e003f58 _dispatch_root_queue_drain_deferred_wlh + 284 24 libdispatch.dylib 0x18e00385c _dispatch_workloop_worker_thread + 720 25 libsystem_pthread.dylib 0x18e1adf9c _pthread_wqthread + 292 26 libsystem_pthread.dylib 0x18e1acce0 start_wqthread + 8
0
0
46
2d
Siri AI (2.0) in MacOS 27 RC - Ends Conversations
While I am liking Siri AI I find that it sometimes will just end a conversation. I can be going back and forth on a topic, and suddenly the input field is gone, and at the bottom of the window it says "Siri ended the conversation". Why does it do this? Once it has, the conversation is just an archive, there is no way to continue the conversation. Starting a new conversation of course forgets everything that came earlier so is pretty useless. I can not understand why Apple would make it work this way. The end point seems random, sometimes it is very quickly after a conversation starts, sometimes it never happens even after days go by.
1
1
832
3d
iOS 27 beta 3/4: Siri AI never enrolls
Device: iPhone 15 Pro iOS: 27.0 beta 4 (same issue on beta 3) Related Feedback: FB23788932, FB23961529 (both marked "More than 10 similar reports", still Open) Siri AI / Apple Intelligence never activate. Extensive testing rules out account/region as the cause — this looks like a broken asset delivery / enrollment pipeline. Symptoms: Console (subsystem com.apple.GenerativeModels) shows repeated calls: isUseCaseAccessNotGrantedSecure: user=501, input=["com.apple.Siri.EnhancedSiriDisablement"] isUseCaseAccessNotGrantedSecure: returning granted (false); no pendingEnrollment for any of [...] ["com.apple.Siri.EnhancedSiriDisablement"] -> false This is consistent across hundreds of calls — the system never attempts enrollment, it just returns false immediately, every time. Settings > General > iPhone Storage shows 3.38GB already allocated to "Apple Intelligence," but the feature never activates — suggesting an incomplete/corrupted asset set rather than missing data entirely. Toggling Wi-Fi off prompts a ~9.5GB "intelligence tools" download. Confirming it produces no progress and no result. Siri language pack downloads get stuck at 100% and never proceed to activation. Search and Siri Suggestions indexing (Settings > Siri & Search) initially shows no percentage, disappears, then reappears days later with a percentage stuck for 24+ hours despite "Last updated: X minutes ago" continuing to refresh — suggesting the background worker is alive but stuck, possibly hitting the same broken asset service. Region-dependent behavior (most useful clue): in a region NOT eligible for Siri AI (Ukraine), legacy Siri (old interface) responds normally to "Hey Siri." In a region eligible for Siri AI (US — tested with a brand-new Apple Account, region set to US, no data restored from backup), "Hey Siri" activates (wake word detection works) but the request hangs indefinitely with no response, and legacy Siri does not answer either. This suggests the system correctly detects eligibility, but there is no fallback to the legacy Siri response pipeline when the region is eligible yet the new Foundation Models assets fail to finish downloading/activating. Already tried (no effect on any of these): Reset Network Settings Reset All Settings Multiple restarts, multiple Wi-Fi networks, cellular data Changing device Language & Region to US Fresh Apple Account created with US region, Payment Method: None, signed in clean (no backup restore) — identical isUseCaseAccessNotGrantedSecure: false result, legacy Siri also silent under this account Steps to reproduce: Update iPhone 15 Pro to iOS 27 beta 3 or 4 via Software Update (not clean install). Settings > Apple Intelligence & Siri — no functional enrollment progress. Toggle Wi-Fi off — download prompt appears, confirming does nothing. Say "Hey Siri" — activates, no response. Has anyone else on iPhone 15 Pro hit this specific isUseCaseAccessNotGrantedSecure / no pendingEnrollment pattern? Any word on whether this is a known/tracked issue for beta 5?
1
0
1.3k
3d
Receiving an on‑screen image from another app via App Intents / Siri (app has no photo library)
I have a photo editing app that owns no photo library. I want a user viewing an image in another app (e.g. Photos) to say "filter this image in MyApp" and have Siri hand that on‑screen image to my intent. Targeting iOS 27. What I've tried, and the result in each case: • App Shortcut + @Parameter var image: IntentFile — Siri resolves my other parameters (a filter AppEnum) by voice, but never binds the image; the run fails. • @AppIntent(schema: .photos.setFilter) with a .photos.asset entity — never routes from Photos. • @AppIntent(schema: .system.open): OpenIntent with a custom AppEntity target — "Open this image in MyApp" just launches the app by name; perform() is never called, and the entity query never runs. My understanding from WWDC26 "Build intelligent Siri experiences with App Schemas" (session 240) and "Discover new capabilities in the App Intents framework" (session 345): • Cross‑app content transfer (Transferable + IntentValueRepresentation) seems limited to system value types (IntentPerson, PlaceDescriptor); IntentFile is not a _SystemIntentValue, so an image can't ride that rail. • Onscreen awareness (NSUserActivity.appEntityIdentifier, View Annotations) appears to expose only the foreground app's own content — which here is Photos, not me. Question: Is there a supported way for a third‑party app to receive another app's on‑screen image (vs. a contact/place) through Siri/App Intents today? If so, which API carries the pixels — an IntentFile parameter, @UnionValue, IntentValueQuery, something else — and what must the source app do to make it available? Or is asking "do X to this image in <third‑party app>" simply not supported yet outside Shortcuts?
4
1
1.2k
4d
Siri AI + Schema .system.open
Since iOS 18, I have an OpenIntent to open documents. For Siri AI, I understood that I need to annotate the entity with @AppIntent(schema: .system.open) for Siri AI to be able to open documents. This is only supported starting with iOS 27. I tried duplicating the intent (one for iOS 27, one for the other versions), however, Xcode complains and says that only one OpenIntent is possible per target entity. How are we supposed to: support Siri AI "open" functionality preserve functionality for older iOS versions ? Thank you
1
2
468
4d
OpenIntent vs .system.open App Schema: Which should be used for opening entities on iOS 27 and later?
I'm trying to understand the intended relationship between OpenIntent and the new .system.open App Intent schema introduced in iOS 27. From the documentation: OpenIntent (available since iOS 16) is described as an intent that opens an associated item. iOS 27 introduces the .system.open schema, which also appears to represent opening an entity or piece of app content. My questions are: For an app that supports iOS 27+, is .system.open intended to replace OpenIntent, or do the two serve different purposes? For apps that support both iOS 26 and iOS 27+, is the recommended approach to have two structs that implement the same opening logic, one with @AppIntent(schema: .system.open) and the other implementing the OpenIntent protocol? Thanks! References: open protocol OpenIntent
3
0
1.6k
4d
Adding MCP and connector support to your own Foundation Models apps
Circling back on the LocalLM Lab arc. With v0.7, we've moved from prompt experimentation into real app development on Apple's Foundation Models local AI. The LocalLM Lab SDK lets you build that same on-device model and MCP client this thread has covered directly into your own app, with real tool and data access (Slack, Todoist, GitHub, Notion, Linear, plus Calendar, Reminders, Contacts and Location). And you can ship your app including through the Mac App Store. This is a big improvement over version 0.6, where the localai-cli toolkit needed LocalLM Lab installed and running. On the other hand, the SDK (LocalLMLabSDKCore) doesn't relay through anything; it links FoundationModels and a real MCP client directly into your own binary and is totally self-contained. The example included in the SDK, Plate Today, has actually been built into a sandboxed test app and verified working, with a signed path to a Mac App Store .pkg (Apple Distribution signing + provisioning profile pipeline). That's "verified signable and sandbox-compatible," to be precise. Entitlements (from personal experience: always a complicated topic): com.apple.security.app-sandbox + com.apple.security.network.client for the app itself, plus the standard personal-information entitlements per connector used (com.apple.security.personal-information.calendars, .addressbook, .location) and matching NS*UsageDescription strings in Info.plist. The one worth flagging specifically: the network entitlement is easy to miss and fails silently rather than throwing. Without it, MCP connections and Weather calls just hang with no error surfaced. OAuth handling requires the app delegate callback (application(_:open:)), not SwiftUI's .onOpenURL. Worth knowing before wiring it up if you're SwiftUI-only. Full entitlements list + SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/sdk.html I hope the availability of the SDK (free, Apache 2.0 license) will give folks further incentive to explore local AI-enabled applications on the Mac. What else would you want to do that the SDK doesn't currently support? File picker? Calendar/Reminders/Contacts edits & writes?
4
1
1.4k
4d
Exploring Apple Silicon + MLX for a persistent local AI companion architecture
I’m developing an independent project in Scotland called Isla Watson. The architecture is built around a simple principle: the model is replaceable; the identity is not. Long-term memory, persistent internal state and identity are designed to remain outside the foundation model, allowing local models to act as replaceable reasoning and language components without resetting the companion. I’m now exploring whether Apple Silicon and MLX could provide the long-term local compute platform for the system — including specialist Mac nodes for reasoning, memory, speech and perception, with distributed inference when larger models are required. A particular area of interest is whether multiple Macs can be used in two complementary ways: as independent specialist agents during normal operation; and as a distributed MLX inference group when a larger model exceeds the capacity of one machine. The first technical study I’d like to establish is a reproducible 1-node → 2-node baseline, measuring model capacity, unified-memory use, time to first token, generation throughput, power consumption, agent concurrency and distributed scaling efficiency. The wider research goal is to keep persistent identity and state independent from whichever foundation model is currently providing language and reasoning. I’d particularly value guidance from anyone working with MLX distributed inference, Thunderbolt/RDMA multi-Mac setups, or local agent architectures. I’ve also posted an architecture-level overview in the MLX GitHub community and have a one-page public brief available for anyone interested in the wider design. https://github.com/ml-explore/mlx/discussions/4482
1
0
307
4d
iOS 27 Beta 1: iPhone 17 reverted to Old Siri instead of New Siri.
My phone no longer shows the waitlist for Siri and has the option to "Try New Siri." I select it, continue, continue and the settings change to "Siri (Beta)" and the waitlist option is no longer there, but when using Siri it's the old pre-Apple Intelligence Siri that activates (little bubble at the bottom) and it does not work. Going to Safari and typing "Siri://" opens the New Siri App, but it says "Siri Update in Progress; Adding support for Siri hasn't completed. Open Settings to check the status." The app does not show up in Spotlight. My phone is done Indexing and all signs point to my phone being enrolled to use the New Siri, but it isn't working at all and still has not shown up. I've tried restarting a few times. Anyone experiencing this too?
9
3
4.3k
1w
iOS 27 + RecognizedText misses lines
I'm using the iOS 18+ RecognizedTextRequest, and with no change of code, iOS 27 gives poorer results: lines are dropped, all the text is not recognized it's not my code, because Live Text has the same issue and of course, back to 26 or 18 solves the issue What changed? Only interesting infos I could isolate: The simulator has not the issue Squashing the image horizontally (80% of its width for example) makes the detection significantly better
1
0
438
1w
PrivateCloudComputeLanguageModel — session.respond hangs for minutes then throws FoundationModels.LanguageModelError -1 wrapping GenerativeFunctionsFoundation.GenerativeError 5040000 (macOS app, not a simulator)
I'm trying to add Private Cloud Compute support to a macOS app and can't get session.respond to succeed. Every call — even a trivial one-line prompt — hangs for one to several minutes ("Thinking" in my UI) and then fails. My setup: macOS 27.2 beta on an M2 MBP Xcode 27.2 The Private Cloud Compute capability is added in Signing & Capabilities, and com.apple.developer.private-cloud-compute is present in the built app's entitlements Removing the entitlement produces the expected error at construction ("Missing entitlement: com.appledeveloper.private-cloud-compute"), confirming it's genuinely being read PrivateCloudComputeLanguageModel().availability reports .available Apple Intelligence is enabled, I'm signed into iCloud with an eligible account/region, and system language is English The failure reproduces with zero tools registered on the session, and with a single isolated request. Relevant code: swift let model = PrivateCloudComputeLanguageModel() let session = LanguageModelSession(model: model, instructions: "You are a helpful assistant.") let result = try await session.respond(to: "What is 17+10?") The error, in full, after several minutes: Error Domain=FoundationModels.LanguageModelError Code=-1 "The operation couldn't be completed. (FoundationModels.LanguageModelError error -1.)" UserInfo={NSLocalizedDescription=The operation couldn't be completed. (FoundationModels.LanguageModelError error -1.), NSMultipleUnderlyingErrorsKey=( "Error Domain=FoundationModels.LanguageModelError Code=-1 "(null)" UserInfo={NSMultipleUnderlyingErrorsKey=(\n "Error Domain=com.apple.GenerativeFunctionsFoundation.GenerativeError Code=5040000 \"(null)\" UserInfo={NSMultipleUnderlyingErrorsKey=(\n)}"\n)}" )} Questions: Is this a known issue with Private Cloud Compute for macOS apps at this point in the beta? Does GenerativeFunctionsFoundation.GenerativeError code 5040000 mean anything specific — a connectivity failure, a capacity/availability issue, something else? Is there a way to get more diagnostic detail than this generic top-level error — a specific log subsystem to check in Console, for instance? Any guidance appreciated — happy to provide a full sysdiagnose or additional repro detail if useful.
Replies
1
Boosts
0
Views
517
Activity
14h
Best Practices for Building AI-Powered Features in iOS Applications
I am exploring different approaches for integrating AI capabilities into modern iOS applications and would like to learn from developers who have built AI-powered experiences. Some areas I am interested in: Best architecture patterns for AI-powered iOS apps Handling API communication securely Managing latency and offline scenarios Protecting user data when working with AI services Designing a reliable user experience around AI-generated responses For developers who have implemented AI features in production apps: What frameworks, architectures, or patterns have worked well for you? Are there any common mistakes you would recommend avoiding when building AI-powered applications on Apple platforms?
Replies
0
Boosts
0
Views
425
Activity
1d
iPhone 16 Pro failing to install new Siri Beta
I am currently on Apple's Dev Beta V4 for iOS 27. The first version I installed was the Dev Beta V2, I am desperate to try out the new Siri AI Beta, but it's just not installing for me. I have the ability to "turn siri off" then "on again" and find I get the 2024 Apple Intelligence version fine. But if I choose to try out the new AI Beta, I'm left with "Adding support for Siri is in progress. Siri will be unavailable until the update is complete." It's been in that state for over 48 hours in Beta 4 and I'm left with the OLD OLD Siri globe from pre-Apple intelligence. Am I being too keen and just not leaving it long enough? Or is there a genuine issue at Apple's end, in regard to getting the new Siri to actually fully install?
Replies
42
Boosts
5
Views
12k
Activity
1d
macOS 27.0 (26A428): Core ML multifunction ML Program is recognized by MLModelAsset but fails to load
Hello, We are seeing what appears to be a regression in the Core ML multifunction ML Program loading path on macOS 27.0. A compiled multifunction ML Program is correctly recognized by MLModelAsset and MLModelStructure, but loading either named function through MLModel fails with an error claiming that the model is not an ML Program. Environment macOS 27.0 Build: 26A428 Apple silicon Mac BABANE 1.0.4, build 16 Application built with the macOS 26.5 SDK Reproduces both inside and outside App Sandbox Approximately 98 GiB of disk space is available Public reproduction BABANE is available from the Mac App Store: BABANE on the App Store Apple engineers can reproduce the issue without receiving a separate model archive: Install BABANE from the App Store on macOS 27.0. Download either available translation model in the app. The model is delivered using Apple-Hosted Background Assets. Trigger model loading by starting a translation. Core ML fails while loading the first named function. The downloadable models are approximately 1.9 GB, so the App Store build is the most practical complete reproduction environment. Model structure The model is a specification-version-9 ML Program containing two functions: infer prefill Core ML correctly recognizes both functions: let asset = try MLModelAsset(url: compiledModelURL) let functionNames = try await asset.functionNames print(functionNames) Output: ["infer", "prefill"] MLModelStructure also returns a .program structure containing both functions. Loading code import CoreML func loadModel( at url: URL, functionName: String? ) throws -> MLModel { let configuration = MLModelConfiguration() configuration.computeUnits = .cpuAndNeuralEngine configuration.functionName = functionName return try MLModel( contentsOf: url, configuration: configuration ) } Loading either function: try loadModel(at: compiledModelURL, functionName: "infer") or: try loadModel(at: compiledModelURL, functionName: "prefill") fails with: `MLModelConfiguration`'s `.functionName` property must be `nil` unless the model type is ML Program. This contradicts the results returned by MLModelAsset and MLModelStructure. Setting functionName to nil is not a workaround. It fails with: This MLModel doesn't support the multi-function description syntax. Unified logging Immediately before the public Core ML error, unified logging reports: E5RT encountered an STL exception. E5RT: <private> (11) Core ML then returns the misleading functionName error. Tests performed We tested: functionName = "infer" functionName = "prefill" functionName = nil .cpuOnly .cpuAndGPU .cpuAndNeuralEngine .all App Sandbox application Non-sandboxed command-line executable Existing .mlmodelc A newly compiled .mlmodelc produced on macOS 27 All named-function combinations fail in the same way. The failure is independent of compute-unit selection and App Sandbox. The source package recompiles successfully on macOS 27, but the newly compiled model still fails to load. As an additional control: A system-provided multifunction ML Program exhibits the same loading failure on this installation. A single-function Core ML model loads successfully. This appears specific to the multifunction model loading path. Documentation The current Core ML documentation still describes MLModelAsset.functionNames as the way to discover functions and MLModelConfiguration.functionName as the way to select one: MLModelConfiguration.functionName MLModelAsset.functionNames We could not find any macOS 27 documentation or release-note entry stating that this behavior changed, that named functions now require a different loading API, or that a new entitlement is required. We found some potentially related reports: Core ML loading crash on macOS 27.0 build 26A428 Historical multifunction model loading crash Core ML/E5RT AOT loading regression with an Apple DTS response None of these reports documents the exact functionName failure described here. Expected behavior A model recognized as a multifunction ML Program should load when MLModelConfiguration.functionName is set to one of the names returned by MLModelAsset.functionNames. Actual behavior MLModel rejects the named function and incorrectly reports that the model is not an ML Program. Questions Is this a known macOS 27.0 regression in the Core ML multifunction loading path? Does MLModelConfiguration.functionName still accept names returned by MLModelAsset.functionNames on macOS 27? Is there a new required loading API, deployment target, SDK, entitlement, or model-packaging rule? Is there a supported workaround other than exporting each function as a separate model? Which diagnostics should we attach to a Feedback Assistant report besides the reproducer, unified logs, sysdiagnose, and exact OS/Xcode builds? Thank you.
Replies
0
Boosts
0
Views
231
Activity
1d
Error in Xcode console
Lately I am getting this error. GenerativeModelsAvailability.Parameters: Initialized with invalid language code: en-GB. Expected to receive two-letter ISO 639 code. e.g. 'zh' or 'en'. Falling back to: en Does anyone know what this is and how it can be resolved. The error does not crash the app
Replies
5
Boosts
2
Views
2.0k
Activity
1d
FoundationModels guided generation: empty token masks and severe slowdowns on macOS 27 betas 5, 6 and 7
Has anyone else hit this? We have a Mac app that uses FoundationModels with @Generable types for structured output. Starting with macOS 27 beta 5 every guided generation request began logging tokenizer errors and long structured requests slowed from seconds to minutes. Beta 6 and beta 7 both still have it. Filed as FB24310823 on August 11 with a full sysdiagnose and log captures, and we have appended evidence from each beta since. The signature is easy to check. Stream the log while your app generates: log stream --predicate 'subsystem == "com.apple.tokengenerationcore"' --style compact On an affected machine the inference service (TGOnDeviceInferenceProviderService, category guided) prints these two lines in matched pairs, thousands of times: Generated an empty mask at recognizer index N allowedTokenIDs is empty. Something is likely wrong with the tokenizer What we measured on beta 7 today: 9,008 of those pairs in about five and a half minutes of scanning. The errors start about one second into the first request after a fresh app launch, so it needs no warmup. Requests that normally finish in 4 to 12 seconds take 77 to 170 seconds or longer. On beta 5 we measured decode at roughly 0.3 tokens per second on the worst requests. Short requests still finish at normal speed but they emit the same errors while they run, and the quality of the structured content they return is degraded. On betas 5 and 6 we also saw repeated asset release errors for instruct_300m.tokenizer and the instruct_3b tokenizer saying the asset is not marked as in use. For what it is worth, a build that ran clean on beta 4 shows the same behavior on beta 5 and later with no app changes, and the same @Generable schema drives both the fast and the slow requests. But we know that does not rule out something on our side, and we would honestly be happy to learn this is our own bug since that would mean we can fix it. So two questions. Is anyone else seeing this since beta 5? And if you spot something we might be doing wrong on our end, sessions we should be recreating, schema patterns that stress the constrained decoder, anything at all, we would really appreciate the feedback. If it does turn out you are hitting the same thing, a Feedback referencing FB24310823 would help a lot. Thanks!
Replies
20
Boosts
1
Views
3.5k
Activity
2d
MLModelAsset(specification:blobMapping:) with mlprogram model: correct predictions but drastically slower inference than compiled .mlmodelc path
I'm distributing an encrypted .mlpackage to my app and want to load it entirely in memory without ever writing decrypted weights to disk. I tried MLModelAsset(specification:blobMapping:) as the path to achieve this, but ran into a significant inference performance gap compared to the compiled code path. What I'm trying to do The encrypted .enc file is a serialized FileWrapper of the full .mlpackage, sealed with AES-GCM. At runtime I decrypt it in memory, deserialize the FileWrapper, extract the spec and weight blob, and load via MLModelAsset: static func loadEncryptedPackage(url: URL, configuration: MLModelConfiguration) async throws -> MLModel { // AES-GCM decryption → decryptedData (full serialized .mlpackage) guard let wrapper = FileWrapper(serializedRepresentation: decryptedData) else { throw ... } guard let (specWrapper, specParent) = findSpecWrapper(in: wrapper), let spec = specWrapper.regularFileContents else { throw ... } var blobs: [URL: Data] = [:] collectBlobs(in: specParent, relativePath: "", excluding: specWrapper, into: &blobs) // keys built as URL(fileURLWithPath: rel), e.g. "weights/weight.bin" let asset = try MLModelAsset(specification: spec, blobMapping: blobs) let model = try await MLModel.load(asset: asset, configuration: configuration) // See observation #3 below — must retain these for the model's lifetime objc_setAssociatedObject(model, &retentionKey, Retainer(spec: spec, blobs: blobs), .OBJC_ASSOCIATION_RETAIN) return model } What I observed Predictions are accurate. The blobs are found, weights are applied, and the model produces correct results. Inference is drastically slower than the compiled code path. The same model loaded via MLModel.compileModel(at:) + MLModel.load(contentsOf:) runs inference much faster on the same device with the same MLModelConfiguration (computeUnits = .all). With MLModelAsset the slowdown is consistent across every prediction call, not just the first one. The spec and blob Data objects must stay alive for the model's lifetime. Without retaining them via objc_setAssociatedObject, inference produces NaN outputs or crashes. This suggests Core ML holds a reference back into those Data buffers beyond the load() call, rather than copying them into its own memory during loading. Using the exact blob URI from the spec as the blobMapping key triggers a compilation error. The spec (inspected via strings on the .mlmodel protobuf) stores blob references as @model_path/weights/weight.bin. When I key the blobMapping with URL(string: "@model_path/weights/weight.bin"), MLModel.load(asset:) throws: compiler error: Encountered an error while compiling a model: validator error: The in-memory ML Program must not have a blob file reference but found a reference to mem://weights/weight.bin. With other key formats (e.g. URL(fileURLWithPath: "weights/weight.bin")), this error does not appear — the model loads and predictions are accurate, but inference is slow as in observation #2. The working alternative (which I want to avoid) Decrypting to a temporary directory, calling MLModel.compileModel(at:), loading from the compiled .mlmodelc, then deleting the temp files produces fast inference. Same model, same device, same configuration. The only difference is the compilation step — and the fact that decrypted weights touch disk, which I want to avoid for security reasons. Questions Is MLModelAsset(specification:blobMapping:) expected to produce inference performance equivalent to loading from a compiled .mlmodelc? If not, is the performance gap fundamental to the API or something that can be addressed? Is there any supported way to load an mlprogram model with external weight blobs entirely in memory and achieve inference performance comparable to the compiled code path — i.e. without writing decrypted model data to disk at any point? The validator error "in-memory ML Program must not have a blob file reference" is a hard block when Core ML successfully resolves the blobs and attempts mlprogram compilation. Is this an intended constraint, and does it mean MLModelAsset(specification:blobMapping:) is not the right API for this use case?
Replies
2
Boosts
1
Views
663
Activity
2d
Crakling with iOS 27
I developed a conversational agent using Gradium and it worked fine using the iPhone's built-in speaker on iOS 18.7 and iOS 26.6. I developed similar conversational agent using the Gemini stack and it also worked fine using the iPhone's built-in speaker on iOS 18.7 and iOS 26.6. Now on iOS 27.0, we hear crackling in the transcripts and responses, with both stacks.
Replies
1
Boosts
0
Views
43
Activity
2d
modelWithContentsOfURL crashed (Apple M4 Pro)
Process: MagiMir2 [5177] Path: /Applications/MagiMir2.app/Contents/MacOS/MagiMir2 Identifier: com.magic.magimir Version: 2.0.2 (2.0.2) Code Type: ARM-64 (Native) Role: Foreground Parent Process: zsh [3124] Coalition: com.apple.Terminal [1945] Responsible Process: Terminal [2656] User ID: 501 Date/Time: 2026-09-16 01:16:23.1769 +0700 Launch Time: 2026-09-16 01:16:21.1311 +0700 Hardware Model: Mac16,7 OS Version: macOS 27.0 (26A428) Release Type: User Thread 19 Crashed:: Dispatch queue: com.apple.coreml.MLModelAssetResourceFactory.modelLoadQueue 0 libsystem_platform.dylib 0x18e1bba50 __bzero + 32 1 libc++.1.dylib 0x18e0beca8 std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>::append(unsigned long, char) + 156 2 CoreML 0x199be7d50 operator>>(IArchive&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>&) + 64 3 CoreML 0x199b57fd4 -[MLLoaderEvent extractAndSetModelDetailsFromArchive:] + 336 4 CoreML 0x199b27c88 +[MLLoader _loadModelFromArchive:configuration:loaderEvent:useUpdatableModelLoaders:error:] + 96 5 CoreML 0x199b25938 +[MLLoader _loadModelFromAssetAtURL:configuration:loaderEvent:error:] + 280 6 CoreML 0x199b256c0 +[MLLoader loadModelFromAssetAtURL:configuration:error:] + 112 7 CoreML 0x199a6f18c -[MLModelAssetResourceFactoryOnDiskImpl modelWithConfiguration:error:] + 120 8 CoreML 0x199bd29b4 __60-[MLModelAssetResourceFactory modelWithConfiguration:error:]_block_invoke + 72 9 libdispatch.dylib 0x18e00a5a0 _dispatch_client_callout + 16 10 libdispatch.dylib 0x18e000490 _dispatch_lane_barrier_sync_invoke_and_complete + 56 11 CoreML 0x199bd287c -[MLModelAssetResourceFactory modelWithConfiguration:error:] + 284 12 CoreML 0x199be8ae8 -[MLModelAssetModelVendor modelWithConfiguration:error:] + 156 13 CoreML 0x199b04a78 -[MLModelAsset modelWithConfiguration:error:] + 116 14 CoreML 0x199b89e94 +[MLModel modelWithContentsOfURL:configuration:error:] + 176 15 MagpieMobile 0x1026dd974 0x10213c000 + 5904756 16 MagpieMobile 0x102710d28 0x10213c000 + 6114600 17 magic_flutter_plugin_magpie 0x100e5c40c closure #1 in MagicFlutterPluginMagpiePlugin.preloadModels(result:) + 192 18 magic_flutter_plugin_magpie 0x100e56134 thunk for @escaping @callee_guaranteed () -> () + 28 19 libdispatch.dylib 0x18dff0a34 _dispatch_call_block_and_release + 32 20 libdispatch.dylib 0x18e00a5a0 _dispatch_client_callout + 16 21 libdispatch.dylib 0x18dff8fc0 _dispatch_lane_serial_drain + 744 22 libdispatch.dylib 0x18dff9abc _dispatch_lane_invoke + 392 23 libdispatch.dylib 0x18e003f58 _dispatch_root_queue_drain_deferred_wlh + 284 24 libdispatch.dylib 0x18e00385c _dispatch_workloop_worker_thread + 720 25 libsystem_pthread.dylib 0x18e1adf9c _pthread_wqthread + 292 26 libsystem_pthread.dylib 0x18e1acce0 start_wqthread + 8
Replies
0
Boosts
0
Views
46
Activity
2d
Siri AI (2.0) in MacOS 27 RC - Ends Conversations
While I am liking Siri AI I find that it sometimes will just end a conversation. I can be going back and forth on a topic, and suddenly the input field is gone, and at the bottom of the window it says "Siri ended the conversation". Why does it do this? Once it has, the conversation is just an archive, there is no way to continue the conversation. Starting a new conversation of course forgets everything that came earlier so is pretty useless. I can not understand why Apple would make it work this way. The end point seems random, sometimes it is very quickly after a conversation starts, sometimes it never happens even after days go by.
Replies
1
Boosts
1
Views
832
Activity
3d
iOS 27 beta 3/4: Siri AI never enrolls
Device: iPhone 15 Pro iOS: 27.0 beta 4 (same issue on beta 3) Related Feedback: FB23788932, FB23961529 (both marked "More than 10 similar reports", still Open) Siri AI / Apple Intelligence never activate. Extensive testing rules out account/region as the cause — this looks like a broken asset delivery / enrollment pipeline. Symptoms: Console (subsystem com.apple.GenerativeModels) shows repeated calls: isUseCaseAccessNotGrantedSecure: user=501, input=["com.apple.Siri.EnhancedSiriDisablement"] isUseCaseAccessNotGrantedSecure: returning granted (false); no pendingEnrollment for any of [...] ["com.apple.Siri.EnhancedSiriDisablement"] -> false This is consistent across hundreds of calls — the system never attempts enrollment, it just returns false immediately, every time. Settings > General > iPhone Storage shows 3.38GB already allocated to "Apple Intelligence," but the feature never activates — suggesting an incomplete/corrupted asset set rather than missing data entirely. Toggling Wi-Fi off prompts a ~9.5GB "intelligence tools" download. Confirming it produces no progress and no result. Siri language pack downloads get stuck at 100% and never proceed to activation. Search and Siri Suggestions indexing (Settings > Siri & Search) initially shows no percentage, disappears, then reappears days later with a percentage stuck for 24+ hours despite "Last updated: X minutes ago" continuing to refresh — suggesting the background worker is alive but stuck, possibly hitting the same broken asset service. Region-dependent behavior (most useful clue): in a region NOT eligible for Siri AI (Ukraine), legacy Siri (old interface) responds normally to "Hey Siri." In a region eligible for Siri AI (US — tested with a brand-new Apple Account, region set to US, no data restored from backup), "Hey Siri" activates (wake word detection works) but the request hangs indefinitely with no response, and legacy Siri does not answer either. This suggests the system correctly detects eligibility, but there is no fallback to the legacy Siri response pipeline when the region is eligible yet the new Foundation Models assets fail to finish downloading/activating. Already tried (no effect on any of these): Reset Network Settings Reset All Settings Multiple restarts, multiple Wi-Fi networks, cellular data Changing device Language & Region to US Fresh Apple Account created with US region, Payment Method: None, signed in clean (no backup restore) — identical isUseCaseAccessNotGrantedSecure: false result, legacy Siri also silent under this account Steps to reproduce: Update iPhone 15 Pro to iOS 27 beta 3 or 4 via Software Update (not clean install). Settings > Apple Intelligence & Siri — no functional enrollment progress. Toggle Wi-Fi off — download prompt appears, confirming does nothing. Say "Hey Siri" — activates, no response. Has anyone else on iPhone 15 Pro hit this specific isUseCaseAccessNotGrantedSecure / no pendingEnrollment pattern? Any word on whether this is a known/tracked issue for beta 5?
Replies
1
Boosts
0
Views
1.3k
Activity
3d
Receiving an on‑screen image from another app via App Intents / Siri (app has no photo library)
I have a photo editing app that owns no photo library. I want a user viewing an image in another app (e.g. Photos) to say "filter this image in MyApp" and have Siri hand that on‑screen image to my intent. Targeting iOS 27. What I've tried, and the result in each case: • App Shortcut + @Parameter var image: IntentFile — Siri resolves my other parameters (a filter AppEnum) by voice, but never binds the image; the run fails. • @AppIntent(schema: .photos.setFilter) with a .photos.asset entity — never routes from Photos. • @AppIntent(schema: .system.open): OpenIntent with a custom AppEntity target — "Open this image in MyApp" just launches the app by name; perform() is never called, and the entity query never runs. My understanding from WWDC26 "Build intelligent Siri experiences with App Schemas" (session 240) and "Discover new capabilities in the App Intents framework" (session 345): • Cross‑app content transfer (Transferable + IntentValueRepresentation) seems limited to system value types (IntentPerson, PlaceDescriptor); IntentFile is not a _SystemIntentValue, so an image can't ride that rail. • Onscreen awareness (NSUserActivity.appEntityIdentifier, View Annotations) appears to expose only the foreground app's own content — which here is Photos, not me. Question: Is there a supported way for a third‑party app to receive another app's on‑screen image (vs. a contact/place) through Siri/App Intents today? If so, which API carries the pixels — an IntentFile parameter, @UnionValue, IntentValueQuery, something else — and what must the source app do to make it available? Or is asking "do X to this image in <third‑party app>" simply not supported yet outside Shortcuts?
Replies
4
Boosts
1
Views
1.2k
Activity
4d
Siri AI + Schema .system.open
Since iOS 18, I have an OpenIntent to open documents. For Siri AI, I understood that I need to annotate the entity with @AppIntent(schema: .system.open) for Siri AI to be able to open documents. This is only supported starting with iOS 27. I tried duplicating the intent (one for iOS 27, one for the other versions), however, Xcode complains and says that only one OpenIntent is possible per target entity. How are we supposed to: support Siri AI "open" functionality preserve functionality for older iOS versions ? Thank you
Replies
1
Boosts
2
Views
468
Activity
4d
OpenIntent vs .system.open App Schema: Which should be used for opening entities on iOS 27 and later?
I'm trying to understand the intended relationship between OpenIntent and the new .system.open App Intent schema introduced in iOS 27. From the documentation: OpenIntent (available since iOS 16) is described as an intent that opens an associated item. iOS 27 introduces the .system.open schema, which also appears to represent opening an entity or piece of app content. My questions are: For an app that supports iOS 27+, is .system.open intended to replace OpenIntent, or do the two serve different purposes? For apps that support both iOS 26 and iOS 27+, is the recommended approach to have two structs that implement the same opening logic, one with @AppIntent(schema: .system.open) and the other implementing the OpenIntent protocol? Thanks! References: open protocol OpenIntent
Replies
3
Boosts
0
Views
1.6k
Activity
4d
Adding MCP and connector support to your own Foundation Models apps
Circling back on the LocalLM Lab arc. With v0.7, we've moved from prompt experimentation into real app development on Apple's Foundation Models local AI. The LocalLM Lab SDK lets you build that same on-device model and MCP client this thread has covered directly into your own app, with real tool and data access (Slack, Todoist, GitHub, Notion, Linear, plus Calendar, Reminders, Contacts and Location). And you can ship your app including through the Mac App Store. This is a big improvement over version 0.6, where the localai-cli toolkit needed LocalLM Lab installed and running. On the other hand, the SDK (LocalLMLabSDKCore) doesn't relay through anything; it links FoundationModels and a real MCP client directly into your own binary and is totally self-contained. The example included in the SDK, Plate Today, has actually been built into a sandboxed test app and verified working, with a signed path to a Mac App Store .pkg (Apple Distribution signing + provisioning profile pipeline). That's "verified signable and sandbox-compatible," to be precise. Entitlements (from personal experience: always a complicated topic): com.apple.security.app-sandbox + com.apple.security.network.client for the app itself, plus the standard personal-information entitlements per connector used (com.apple.security.personal-information.calendars, .addressbook, .location) and matching NS*UsageDescription strings in Info.plist. The one worth flagging specifically: the network entitlement is easy to miss and fails silently rather than throwing. Without it, MCP connections and Weather calls just hang with no error surfaced. OAuth handling requires the app delegate callback (application(_:open:)), not SwiftUI's .onOpenURL. Worth knowing before wiring it up if you're SwiftUI-only. Full entitlements list + SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/sdk.html I hope the availability of the SDK (free, Apache 2.0 license) will give folks further incentive to explore local AI-enabled applications on the Mac. What else would you want to do that the SDK doesn't currently support? File picker? Calendar/Reminders/Contacts edits & writes?
Replies
4
Boosts
1
Views
1.4k
Activity
4d
Exploring Apple Silicon + MLX for a persistent local AI companion architecture
I’m developing an independent project in Scotland called Isla Watson. The architecture is built around a simple principle: the model is replaceable; the identity is not. Long-term memory, persistent internal state and identity are designed to remain outside the foundation model, allowing local models to act as replaceable reasoning and language components without resetting the companion. I’m now exploring whether Apple Silicon and MLX could provide the long-term local compute platform for the system — including specialist Mac nodes for reasoning, memory, speech and perception, with distributed inference when larger models are required. A particular area of interest is whether multiple Macs can be used in two complementary ways: as independent specialist agents during normal operation; and as a distributed MLX inference group when a larger model exceeds the capacity of one machine. The first technical study I’d like to establish is a reproducible 1-node → 2-node baseline, measuring model capacity, unified-memory use, time to first token, generation throughput, power consumption, agent concurrency and distributed scaling efficiency. The wider research goal is to keep persistent identity and state independent from whichever foundation model is currently providing language and reasoning. I’d particularly value guidance from anyone working with MLX distributed inference, Thunderbolt/RDMA multi-Mac setups, or local agent architectures. I’ve also posted an architecture-level overview in the MLX GitHub community and have a one-page public brief available for anyone interested in the wider design. https://github.com/ml-explore/mlx/discussions/4482
Replies
1
Boosts
0
Views
307
Activity
4d
Problems
I am using an iPhone 16 Pro with iOS 27 Public Beta. In the Netherlands, the Apple Intelligence option has disappeared from the settings.
Replies
0
Boosts
0
Views
489
Activity
6d
iOS 27 Beta 1: iPhone 17 reverted to Old Siri instead of New Siri.
My phone no longer shows the waitlist for Siri and has the option to "Try New Siri." I select it, continue, continue and the settings change to "Siri (Beta)" and the waitlist option is no longer there, but when using Siri it's the old pre-Apple Intelligence Siri that activates (little bubble at the bottom) and it does not work. Going to Safari and typing "Siri://" opens the New Siri App, but it says "Siri Update in Progress; Adding support for Siri hasn't completed. Open Settings to check the status." The app does not show up in Spotlight. My phone is done Indexing and all signs point to my phone being enrolled to use the New Siri, but it isn't working at all and still has not shown up. I've tried restarting a few times. Anyone experiencing this too?
Replies
9
Boosts
3
Views
4.3k
Activity
1w
iOS 27 + RecognizedText misses lines
I'm using the iOS 18+ RecognizedTextRequest, and with no change of code, iOS 27 gives poorer results: lines are dropped, all the text is not recognized it's not my code, because Live Text has the same issue and of course, back to 26 or 18 solves the issue What changed? Only interesting infos I could isolate: The simulator has not the issue Squashing the image horizontally (80% of its width for example) makes the detection significantly better
Replies
1
Boosts
0
Views
438
Activity
1w
FoundationModels.LanguageModelError error -1 on visionOS simulator
I am getting FoundationModels.LanguageModelError error -1 when trying to use Foundation Models on the visionOS 27 simulator. It works on the iOS 27 simulator. Is this a bug with the visionOS 27 simulator specifically?
Replies
1
Boosts
0
Views
523
Activity
1w