Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

Vision face landmarks shifted on iOS 26 but correct on iOS 18 with same code and image
I'm using Vision framework (DetectFaceLandmarksRequest) with the same code and the same test image to detect face landmarks. On iOS 18 everything works as expected: detected face landmarks align with the face correctly. But when I run the same code on devices with iOS 26, the landmark coordinates are outside the [0,1] range, which indicates they are out of face bounds. Fun fact: the old VNDetectFaceLandmarksRequest API works very well without encountering this issue How I get face landmarks: private let faceRectangleRequest = DetectFaceRectanglesRequest(.revision3) private var faceLandmarksRequest = DetectFaceLandmarksRequest(.revision3) func detectFaces(in ciImage: CIImage) async throws -> FaceTrackingResult { let faces = try await faceRectangleRequest.perform(on: ciImage) faceLandmarksRequest.inputFaceObservations = faces let landmarksResults = try await faceLandmarksRequest.perform(on: ciImage) ... } How I show face landmarks in SwiftUI View: private func convert( point: NormalizedPoint, faceBoundingBox: NormalizedRect, imageSize: CGSize ) -> CGPoint { let point = point.toImageCoordinates( from: faceBoundingBox, imageSize: imageSize, origin: .upperLeft ) return point } At the same time, it works as expected and gives me the correct results: region is FaceObservation.Landmarks2D.Region let points: [CGPoint] = region.pointsInImageCoordinates( imageSize, origin: .upperLeft ) After that, I found that the landmarks are normalized relative to the unalignedBoundingBox. However, I can’t access it in code. Still, using these values for the bounding box works correctly. Things I've already tried: Same image input Tested multiple devices on iOS 26.2 -> always wrong. Tested multiple devices on iOS 18.7.1 -> always correct. Environment: macOS 26.2 Xcode 26.2 (17C52) Real devices, not simulator Face Landmarks iOS 18 Face Landmarks iOS 26
1
0
559
5h
Codex GPT-5.5 got deprecated, no other models selectable in xcode
Oldest model of 5.5 with codex is deprecated by chatgpt but I only see 5.5 as the highest available model selectable in the dropdown in intelligence settings in Xcode when selecting codex I have the ChatGPT pro account and login to codex in xcode via email/password and not the API key I have already tried login out, quitting xcode, removing codex agent and reinstalling it, so far nothing works
3
0
638
10h
iPhone 18 Pro: First Core ML load on Neural Engine up to 65× slower than on iPhone 17 Pro
On iPhone 18 Pro (A20 Pro), the first MLModel load of a model on the Neural Engine can take minutes where an iPhone 17 Pro needs seconds. Cached loads are instant, and once loaded the model runs faster than on the iPhone 17 Pro, so only the on-device specialization is affected. This looks like the same problem as A20 pro devices take too long on CoreML model specialization and iPhone 18 pro takes very long time to load models. The "loading" time there is likely this specialization too: it happens inside MLModel(contentsOf:) for an already compiled .mlmodelc. I can reproduce it reliably with attention in the form Apple recommends for the Neural Engine (split einsum, as in Deploying Transformers on the Apple Neural Engine). In this example most of the extra time comes from the softmax over the key axis, which in that layout is the channel axis. Other ops and model types may well be affected too; this is simply the smallest case we could build that shows the problem clearly. The repro uses four self-attention layers (10 heads × 64, 4096 tokens, fp16) built with coremltools and loaded cold with .cpuAndNeuralEngine. MLComputePlan places every op on the Neural Engine on all three devices. iOS 27.0.1 (24A446) on both iPhones: iPhone 18 Pro iPhone 17 Pro M3 Pro Mac cold predict cold predict cold predict Split-einsum attention, 4 layers 53.4 s 21.6 ms 0.8 s 111.4 ms 1.0 s 105.8 ms Q.K einsum only 5.1 s 24.0 ms 1.2 s 36.8 ms 1.4 s 41.8 ms Q.K einsum + softmax 36.1 s 61.5 ms 1.7 s 141.0 ms 2.0 s 128.3 ms Scores keys-last (softmax last axis) 10.5 s 109.0 ms 0.6 s 198.4 ms 0.7 s 180.2 ms scaled_dot_product_attention 1.4 s 140.4 ms 0.4 s 184.9 ms 0.5 s 148.1 ms Cached loads take 0.01–0.03 s everywhere. What I observed: Compile time grows linearly with the number of attention layers. Every alternative attention layout I tried that compiles quickly runs 5–6× slower on the A20 Pro, so I found no workaround that keeps its speed. During the slow load, ANECompilerService runs almost entirely on the efficiency cores. Requesting a higher QoS, MLOptimizationHints (.fastPrediction, reshapeFrequency), computeUnits = .all and dense instead of palettized weights don't help. Loading several models in parallel additionally gets ANECompilerService killed by jetsam (per-process-limit). The specialization cache doesn't survive reinstalling the app. In our app (Stable Diffusion XL), each UNet chunk with attention takes 110–150 s instead of 6–16 s, so the first launch takes about 25 minutes in the foreground, and in our tests again after every reinstall. The same networks converted to Core AI specialize normally on this device (see our Core AI porting notes). Filed as FB25106520 with a self-contained repro: a coremltools script, an iOS app and a Mac runner, plus results from all three devices. Environment: iPhone 18 Pro (iPhone19,2) and iPhone 17 Pro (iPhone18,1) on iOS 27.0.1 (24A446), M3 Pro on macOS 27.0.1 (26A434), Xcode 27.0, coremltools 8.3.0. Is this a known A20 Pro issue, and is there a recommended way to avoid it until it's fixed? Falling back to the GPU or CPU isn't a good option for us: it avoids the slow load, but gives up the Neural Engine's efficiency, which matters for on-device generation on a phone, and its speed (a full SDXL UNet step on the iPhone 18 Pro takes 509 ms on the Neural Engine vs 1,639 ms with .cpuAndGPU). We'd also like to hear from others which ops their slow first loads involve, to see whether this is specific to attention and softmax or broader.
8
5
1.5k
1d
What signal should drive fallback for PrivateCloudComputeLanguageModel?
I'm building an app that uses PrivateCloudComputeLanguageModel as the primary inference tier with SystemLanguageModel as the fallback. The app is entitled (com.apple.developer.private-cloud-compute, granted and provisioned) and generations serve normally. My question is how a client should decide to fall back because in extended measurement, no public signal ever reflects the blocked state I actually hit. What I measured (macOS 27.0 beta, 26A5416b / Xcode 27 beta 27A5237l, entitled signed bundle constructing PrivateCloudComputeLanguageModel directly): Serving stopped mid-run with no leading signal: request N served normally (1.4 s), request N+1 threw LanguageModelError.rateLimited 494 ms later, at cumulative generation 786 for the day. 100% served → 100% refused between consecutive calls. Every quota signal read healthy the entire time: before, during, and after the block. Across 1,517 readings in a single day: quotaUsage.status = belowLimit, isApproachingLimit = false, isLimitReached = false, resetDate = nil, availability = .available. A preflight on these APIs cannot see the condition. The refusal is enforced locally after first contact: rejections return in ~230 ms vs ~0.9–1.4 s for served calls, so the client appears to cache the verdict rather than ask the server per-request. The trigger is a cumulative ledger, not a request rate: 501 generations at 33/min in one 15-minute sitting was fine, and a later arm sustained 39.7/min; two bursts of 16 concurrent at 5.0 and 5.2 req/s served 32/32; the count that tripped survived a process restart and a 4.9-hour idle gap. But it's not a fixed daily number either. 501 fast was fine earlier the same day; the trip came 285 requests later. A rolling window on the order of hours-to-a-day is consistent with this, but nothing here measures its length. Recovery: still blocked at +41 minutes (probes at +1/2/5/10/20/40 min all refused); fully recovered by +20 h with no intervention and no upgrade. Next day served normally from the first request. quotaLimitReached never occurred: not once in ~800 generations plus the blocked period. The wall is typed as the transient error while carrying what the documentation describes as daily quota semantics ("a person either waits for their usage quota to refresh or they upgrade"). limitIncreaseSuggestion is presence-constant: nil at process start, non-nil on every reading after first PCC contact (identical while fully serving and while fully blocked) so its presence can't gate an upsell affordance. The same signals-read-healthy-while-refusing divergence also reproduces against the developer-tool pool (fm serve), which I've reported separately (FB24273854 covers quota exhaustion surfacing there as a generic server_error/500 while /health reports the model available). Questions: Is attempt-and-classify the intended contract? Given that no preflight can observe the blocked state, should a client simply issue the request, treat the typed error as authoritative, and route to SystemLanguageModel? And is the ~230 ms local fail-fast on the blocked path contractual (cheap and safe to probe) or incidental? This is the one that decides how I ship; the rest are diagnostics behind it. What does quotaUsage actually track, and at what granularity? I have driven the entitled app-tier path to a hard block and the developer-tool pool to exhaustion, and no field ever moved. Is there any consumption pattern that moves isApproachingLimit / isLimitReached / resetDate? If the intended answer is "only the per-person daily quota, which these volumes never approached," what is the wall I am hitting at ~786 cumulative, and why does it surface as rateLimited? Should rateLimited and quotaLimitReached drive different client behavior — and which one is the daily allowance in practice? The documentation distinguishes rate limiting ("wait a period and retry") from daily exhaustion ("wait for refresh or upgrade"), but what I observe is the transient-typed error carrying the multi-hour ledger semantics. Concretely: what retry cadence is recommended after rateLimited (my measured recovery horizon was somewhere between 41 minutes and 20 hours. My current design stays on the on-device model and re-probes PCC at a low fixed interval rather than per-request)? And under what condition is resetDate ever populated, given it was nil even while blocked? (Smaller, design guidance): my app can generate a few hundred requests as one feature batch (quiz generation over a user's imported document). Measured: 501 in a sitting was fine, cumulative 786 in a day was not. Since this allowance belongs to the person and is shared with every Apple Intelligence feature, is a several-hundred-request batch a reasonable use of it, or should features like this generate on demand? (I'm aware of the existing feature request for richer quota reporting (FB23378161); this is a narrower design question.) I can attach the measurement driver and timestamped JSONL logs. The divergence is reproducible on a fresh day, though reaching the wall took ~800 cumulative generations.
5
1
2.1k
2d
iOS 27 + RecognizedText misses lines
I'm using the iOS 18+ RecognizedTextRequest, and with no change of code, iOS 27 gives poorer results: lines are dropped, all the text is not recognized it's not my code, because Live Text has the same issue and of course, back to 26 or 18 solves the issue What changed? Only interesting infos I could isolate: The simulator has not the issue Squashing the image horizontally (80% of its width for example) makes the detection significantly better
6
5
1.8k
3d
Major iOS 27 Regression in '.accurate' Model for VNRecognizeTextRequest
Hello, With the update from iOS 26 to iOS 27 I have noticed significant regressions and degraded performance when using the Vision Framework for OCR via 'VNRecognizeTextRequest'. The degraded performance is only applicable to the .accurate model, the .fast model is identical between iOS 26 and iOS 27. I have an app that uses this framework extensively for extracting information from the video feed of professional cinema cameras. This runs using en-US, language correction off with a specific region of interest. On iOS 26 everything works reliably as it has for the past few years, with the exact same codebase on iOS 27 many strings that were previously detected are no longer detected, some strings return as if they've been read upside down (ISO1280 is returned as 08ZLOSI) and there are a lot of Cyrillic look alikes mixed into Latin text, there also appears to be a lot more junk characters tacked onto the end of numbers ('12ł' is returned when previously it was just '12'). You often see whole line duplicates too where the same text is recognised twice, once as a single line with all strings combined and once as individual strings. The iOS 27 Simulator doesn't show any of this. I only saw it on real iPads, if you only test in the Simulator you'll miss it. To put numbers on it, I built a test set of about 26,500 frame grabs and ran them through 'VNRecognizeTextRequest' on iPadOS 18, 26 and 27. With the version of my app that's on the App Store today, iOS 26 reads 99.7% of values correctly. On iOS 27 that drops to 65%. I've resolved this issue now and have been able to retain the level of performance that existed on iOS 26, this involves various preprocessing steps to the images as well as a significant amount of validation after the Vision Framework returns its results. These fixes took around 3.5 Days of work which is incredibly frustrating. I would much rather use my time building new features instead of fixing regressions introduced by poorly validated new models that have been shipped in iOS 27. What frustrates me most is this change wasn't announced anywhere and there's no way to opt out. The revision number didn't change so there's no way to keep using the model that worked. If this had shipped as VNRecognizeTextRequestRevision4 apps could have stayed on VNRecognizeTextRequestRevision3 until they'd had a chance to test. Or permanently stayed on 'VNRecognizeTextRequestRevision3' as obviously the new version performs significantly worse. Instead an app that worked fine on iOS 26 now gets wrong text back on iOS 27 with no error, no warning and nothing in the release notes. For a framework people rely on for real work that's a significant regression. It seems that other developers have also come across this issue: https://developer.apple.com/forums/thread/845452 For technical reference, if you diff System/Library/PrivateFrameworks/TextRecognition.framework between iOS 26.5 (23F77) and iOS 27.0 (24A434) you uncover a number of changes. New models on iOS 27 seem to use an '_e5' variant. The text detector in iOS 26 references training run 'yyzn76u6fs' with 88560 iterations, iOS 27 references training run 'hvcthxumvz' with only 10116 iterations. Perhaps an 8x reduction in iterations has something to do with the poor performance in iOS 27? Further, the orientation model changes from iOS 26 'cr_orientation_model_v1' with 1858472 iterations vs. iOS 27 'cr_orientation_model_v1_e5' with 671065 iterations. The Latin and Cyrillic Recogniser changed model entirely from 'cr_tr_model_latincyrillic_v3' on iOS 26 to 'cr_tr_model_latincyrillic_v3_e5' on iOS 27. It seems these 'e5' models are optimised for the neural engine. Is there any way to get the iOS 26 models/behaviour back or plans to back deploy the old model in iOS 27.1 or iOS 27.2? Can future model changes like this come with a new revision number? What testing process does the Vision Team use to validate models as clearly something was missed here? Thanks!
0
0
260
3d
iOS 27: ML Program pipeline with data-dependent shapes (NMS / gather / non_zero) fails on every compute unit and allocates ~2.8 GB at load
A Core ML model that we used in production on iOS 26 no longer works on iOS 27.0.1. Prediction now fails on every compute unit. Loading it with .all also increases the process footprint by about 2.8 GB, so on 6 GB devices jetsam kills the app shortly after launch (per-process-limit). Environment iPhone 13 Pro (iPhone14,2), iOS 27.0.1 (24A446): the memory spike and the prediction failure both reproduce. The app is killed about 1.5 s after launch. iPhone 15 Pro, iOS 27.0.1 (24A446): the memory spike reproduces (about 3 GB). The app is not killed because this device's limit is higher. macOS 27.0.1 host, small Swift CLI using Core ML: the prediction failure reproduces. The same model and app build worked on iOS 26.x. On an iPhone 15 Pro running iOS 26.6.1, the app peaked at about 570 MiB. Model A compiled .mlmodelc (from an .mlpackage). It is a pipeline of two ML Programs. The input is a fixed-size 640×480 image. model 0: a YOLO-style instance segmentation network converted from TensorFlow. It includes non_maximum_suppression over 1500 candidate boxes. The NMS is followed by range_1d → gather with data-dependent [?] shapes. There is a zero-sized intermediate tensor (tensor<fp16, [1500, 0]>). Results are written with scatter_nd into a fixed-size output [1, 300, 38], plus prototype masks [1, 120, 160, 32]. model 1: post-processing converted from PyTorch. It runs non_zero over the 300 candidates. After that it works on dynamic [?, 1, 480, 640] tensors (upsample_bilinear, greater, cast, mul), then uses tile and reduce_argmax. Symptom 1: prediction fails on all compute units The failure is the same with .all, .cpuAndNeuralEngine, .cpuAndGPU and .cpuOnly, with a real camera image as input: Error Domain=com.apple.CoreML Code=0 "Failed to evaluate model 0 in pipeline" -> "Unable to compute the asynchronous prediction using ML Program. It can be an invalid input data or broken/unsupported model." -> "E5RT: Submit Async failed for [6:1]: Unable to successfully execute plan. @ ExecuteSync (11)" On macOS 27, the following is logged when the model loads: MIL program has non-constant (dynamic) shapes for external input but FlexibleShapeInformation attribute is missing. E5 function will be produced with unknown input shapes and cannot be run without reshaping first. Got input with flexible dimension but no default shapes provided. Will use 1 Symptom 2: about 2.8 GB allocated at load This is phys_footprint right after MLModel(contentsOf:configuration:). The model was loaded alone in a fresh process on an iPhone 13 Pro running iOS 27.0.1. The baseline before loading was 13 MiB. .all: 2858 MiB .cpuAndNeuralEngine: 2857 MiB .cpuAndGPU: 2515 MiB .cpuOnly: 62 MiB The other models in the same app stay under about 520 MiB on iOS 27 (load plus one prediction each). They include object detection, pose estimation and a ViT that runs on the ANE. Another model in the app with data-dependent shapes, a keypoint detector, still loads and predicts correctly on iOS 27, so not every dynamic-shape model is affected. In the field, the JetsamEvent report shows per-process-limit at 3072 MiB. A diskwrites_resource report shows the heaviest stack under Core ML model load → Espresso → MetalPerformanceShadersGraph. Current workaround Loading this model with .cpuOnly on iOS 27 avoids the memory spike, so the app no longer crashes. Prediction still fails, though, so the model can't be used on iOS 27. Questions Is this a known iOS 27 regression in how E5RT handles data-dependent shapes inside ML Program pipelines (NMS, gather, non_zero, zero-sized tensors)? What is the recommended way to export a model like this for iOS 27? Would adding flexible-shape information help, or should NMS and mask post-processing be moved out of the model? Is it expected that unbounded dynamic shapes cause a worst-case allocation of several GB at load time with GPU or ANE compute units? I can't share the model itself, but I'm happy to provide more details (full MIL op listing, logs, sysdiagnose) through Feedback Assistant if that helps.
2
0
412
3d
iPhone 18 pro takes very long time to load models
I think this could be relate to previous post here https://developer.apple.com/forums/thread/848590 But in my testing it seems like most of the time spent was in loading but not compilation. Our app has multiple models but so far it seems like one of them has this issue. The model itself was only 2.2MB, but somehow loading took 50s after compilation? And on 17e it only took 2s to load. But I cannot attach the model here so please advice.
2
2
1k
4d
Core ML memory usage is dramatically higher with an Xcode 27 build on iOS 27
I’m seeing a major change in reported memory usage (and eventual termination due to memory pressure) when running a Core ML workload built with Xcode 27 on iOS/iPadOS 27. The source code, model files, and MLModelConfiguration are unchanged. Only the Xcode/SDK version used to build the app differs. On the same iPad running iPadOS 27: Xcode 26 build: model loading and prediction complete normally, with a relatively small reported application footprint. Xcode 27 build: the application footprint grows continuously as models are loaded and can exceed 5 GB. The app is eventually terminated unless models are unloaded very aggressively or the increased-memory-limit entitlement is used. I also tested an Xcode 27 build on a device running iOS 26. Its reported peak was only around 300 MB. This suggests the change requires both an Xcode 27-linked binary and the iOS 27 runtime. The workload consists of several compiled Core ML models using .cpuAndNeuralEngine. Loading models sequentially instead of concurrently does not materially change the final footprint. Releasing each MLModel after use does reduce it, so this appears to be model or Neural Engine residency being charged to the application rather than a conventional heap leak. I noticed that the iOS 27 release notes mention Neural Engine memory now being attributed to the application instead of the system. However, I’m unclear about the practical consequences of that change. If the same Neural Engine resources were already physically resident on iOS 26, I would have expected them to contribute to system memory pressure even when they were not attributed directly to the application. Instead, the older configuration runs comfortably, while the Xcode 27/iOS 27 combination approaches or crosses the application’s per-process memory limit. A few additional observations: The problem is more likely to occur after Core ML has already compiled and specialized the models. Cached model loading is much faster and the footprint grows quickly. The first uncached run can survive model preparation because specialization spaces the loads farther apart. Under Instruments, the app often does not terminate, presumably because profiling slows the workload enough to change the peak. os_proc_available_memory() decreases in line with the newly reported footprint. With the increased-memory-limit entitlement, the workload completes, but the reported footprint still reaches several gigabytes. Has anyone else observed a large Core ML memory increase specifically with an Xcode 27 build running on iOS 27? In particular, I’m trying to understand: Is this purely a change in how existing Neural Engine memory is accounted for, or does the new runtime also retain or allocate more memory? Is the new accounting used for the application’s jetsam/per-process memory limit? Is this behavior intentionally gated by the linked SDK version? That would explain why an Xcode 26 build behaves differently on the same iOS 27 device. Should applications now treat the Neural Engine residency of every loaded MLModel as part of their process-memory budget and unload models accordingly? Are there recommended APIs or Core ML loading strategies for controlling this residency? Any confirmation that others are seeing the same Xcode 27/iOS 27 behavior—or clarification of the intended memory-accounting model—would be very helpful.
2
3
1.4k
4d
A20 pro devices take too long on CoreML model specialization
Hi, I'm a ML developer specializing in local audio related models. I upgraded from a 16 pro max to an 18 pro max. I'm not sure what the reason is, but model specialization (happens only on the first load, cpu and neural engine compute units) is sometimes 2x - 5x longer on the 18 pro max vs the 16 pro max. Both devices are on iOS 27. For example, I have a stem separation model running, and the 18 pro max took 171 seconds (including inference) and the 16 pro max took 37 seconds (including inference). 18 pro max inference is ~55x RTFx and the 16 pro max is ~40 RTFx for this particular model, but I've noticed it as well in TTS models etc. Please look into this, I know the 18 pro max has double the neural engine cores, so I'm not sure if the complexity makes specialization take longer. Subsequent loads are great, same high rtfx for the 18 PM (only 0.5s load), and this is where it beats the 16 PM handily. I attached 2 images showing the 18 PM on a first load (model specialization likely) vs a subsequent run. Please look into this. (Note this is not just my implementation, I tested a range of different models, all with the same outcome).
5
6
1.6k
4d
Adding MCP and connector support to your own Foundation Models apps
Circling back on the LocalLM Lab arc. With v0.7, we've moved from prompt experimentation into real app development on Apple's Foundation Models local AI. The LocalLM Lab SDK lets you build that same on-device model and MCP client this thread has covered directly into your own app, with real tool and data access (Slack, Todoist, GitHub, Notion, Linear, plus Calendar, Reminders, Contacts and Location). And you can ship your app including through the Mac App Store. This is a big improvement over version 0.6, where the localai-cli toolkit needed LocalLM Lab installed and running. On the other hand, the SDK (LocalLMLabSDKCore) doesn't relay through anything; it links FoundationModels and a real MCP client directly into your own binary and is totally self-contained. The example included in the SDK, Plate Today, has actually been built into a sandboxed test app and verified working, with a signed path to a Mac App Store .pkg (Apple Distribution signing + provisioning profile pipeline). That's "verified signable and sandbox-compatible," to be precise. Entitlements (from personal experience: always a complicated topic): com.apple.security.app-sandbox + com.apple.security.network.client for the app itself, plus the standard personal-information entitlements per connector used (com.apple.security.personal-information.calendars, .addressbook, .location) and matching NS*UsageDescription strings in Info.plist. The one worth flagging specifically: the network entitlement is easy to miss and fails silently rather than throwing. Without it, MCP connections and Weather calls just hang with no error surfaced. OAuth handling requires the app delegate callback (application(_:open:)), not SwiftUI's .onOpenURL. Worth knowing before wiring it up if you're SwiftUI-only. Full entitlements list + SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/sdk.html I hope the availability of the SDK (free, Apache 2.0 license) will give folks further incentive to explore local AI-enabled applications on the Mac. What else would you want to do that the SDK doesn't currently support? File picker? Calendar/Reminders/Contacts edits & writes?
7
1
4.2k
6d
Model Guardrails Too Restrictive?
I'm experimenting with using the Foundation Models framework to do news summarization in an RSS app but I'm finding that a lot of articles are getting kicked back with a vague message about guardrails. This seems really common with political news but we're talking mainstream stuff, i.e. Politico, etc. If the models are this restrictive, this will be tough to use. Is this intended? FB17904424
11
5
3.1k
6d
Framing an object in real time
Hi, I am currently working on a project which involves giving directions to the user telling them what they should do to frame a given object (also only doing this if sees the object which can be many things, so it needs detection too) so that it can be optimally in view of the camera to be classified. I have figured the classification, but not the real time guidance and detection. I tried training a model that featured all of the different movements ("move up", "move down", etc.). It featured 30 different objects with ten crops per image, and each class featured all 30 images and an even amount per class. It had over 1500 images in total, and I trained using createml and core ml. But, they were very inaccurate in my testing. I also tried using foundation models for this, but it also didn't work. I am quite new to integrating ai/ ml models into a project, and I am unsure of where to go next. If anyone has any suggestions, I would be really appreciative.
0
0
332
1w
SiriAI Voice my app
Policy or no policy I think its clear if Apple wants to have a premium SiriAI experience its gonna need to allow apps to use the Siri AI Voice engine Developers like me want the privacy/no cloud voice engine service that a SiriAI Voice would provide Seemless conversations with versions of SiriAI Voices let's users know they're still in the SiriAI ecosystem but engaged with a developer app The "hey Siri, ask LinkedIN Hoffman to .." is stunningly haphazard moat gate that Apple can resolve and still keep control with a simple App License so it know every app that uses a SiriAI Voice App engine. I've been waiting over a year for Apple to figure this out. This is an unforced error. This is not a threat its an opportunity for Apple ? Is this a Driver or CoreOS issue? WTF?
0
0
339
1w
Dutch Siri identifies my App Shortcut but requests Notes or falls back to Home/Intercom — FB25058524
I need a supported way for Dutch Siri to invoke Elegant Cook's built-in grocery actions directly, without requiring users to create personal shortcuts. The installed TestFlight version is 1.5.3 (32), on an iPad Air (5th generation). The owner reports iPadOS 26.6.1; the diagnostic archive identifies build 23G83. Siri is Dutch, with nl_BE runtime assets. Use with Siri is enabled and spoken responses are preferred. The main app uses App Intents and AppShortcutsProvider with three actions: add a grocery, read the list, and check off a grocery. The voice actions use scalar String parameters with requestValueDialog and return ProvidesDialog results. The phrases use the applicationName token and have packaged English and Dutch localizations. We do not register Notes or Home intents. Observed behavior: “Hé Siri, voeg boodschappen toe in Elegant Cook” and “Hé Siri, lees mijn boodschappenlijst in Elegant Cook” ask whether to use Elegant Cook or Notes. After selecting Elegant Cook, adding and reading work by voice. “Hé Siri, vink een boodschap af in Elegant Cook” and the registered app-name-first equivalent “Hé Siri, Elegant Cook boodschappen afvinken” instead produce the instruction to download Apple's Home app to use Intercom. “Hé Siri, open Elegant Cook” opens the installed app correctly. The signed build contains extracted App Intents metadata, compiled Dutch/English nlu.appintents assets and successful phrase-training output. These build checks do not establish successful device routing or App Shortcuts Preview validation. For one captured check-off invocation, local diagnostic analysis identified this sequence: the recognized utterance includes the correct app name; App Shortcuts are enabled; the intended CheckElegantCookGrocery action is a Siri candidate; app disambiguation then fails and explicitly redirects to Home/Intercom before the grocery operation. This trace covers check-off only. It does not prove that the add/read chooser has the same cause, or establish a definitive Apple framework bug. The private bug report is FB25058524. It includes selected evidence and source code. A full sysdiagnose has not been uploaded, and the isolated Swift example is not a compiled or validated reproducing Xcode project. We develop on Windows and use EAS cloud builds, so local Xcode App Shortcuts Preview is unavailable. The code-level support form currently directs us to this forum when we cannot provide a test project demonstrating the issue. Could an App Intents/DTS engineer advise: Which supported registration or metadata correction should we check when Siri identifies an explicitly named App Shortcut but chooses another domain before perform()? What targeted diagnostic evidence can distinguish an app configuration problem from a Siri routing defect without local App Shortcuts Preview? Is there a supported workaround for direct Dutch voice invocation on this OS build, without personal shortcuts or private system settings? We can prepare an isolated native sample project if required, but will clearly distinguish source preparation, cloud compilation and actual device reproduction. We do not want to claim a reproduction that has not been verified.
0
0
421
1w
iOS 27: concurrent Core ML predictions on the Neural Engine are serialized
Environment iPhone 17 Pro Max, iOS 27.0.1 ([build]) Reference: MacBook Pro (M3 Pro), macOS 26.4.1 ML Program model, MLComputeUnitsCPUAndNeuralEngine. A single prediction takes about 2.8 ms on the ANE. Description On iOS 27, Core ML predictions submitted to the Neural Engine from multiple threads at the same time no longer overlap. The ANE runs them strictly one after another. Adding threads gives no extra throughput, and per-call latency grows linearly with the number of requests in flight. The same code still gets a clear concurrency gain on macOS 26.4.1, and our app's ANE-parallel path on this iPhone became noticeably slower after the iOS 27 update. On iOS 27, latency is about N × single-call time, which means requests are queued and never overlap. Running two different models concurrently on the ANE shows the same behavior. What we tried, with no change in result A separate MLModel instance per thread. The async predictionFromFeatures:options:completionHandler: API. Questions Is this serialization an intended change in iOS 27? Is there a supported way to get concurrent ANE execution back?
0
0
462
1w
iOS 27: English sentence embedding model unavailable on physical devices
I’m seeing NLEmbedding.sentenceEmbedding(for: .english) return nil on physical devices running iOS 27, both in debug builds under Xcode and in production. The same failure has been observed across thousands of users on iOS 27. One affected device runs iOS 27.0.1 (24A446). Minimal example with diagnostics: import NaturalLanguage let language = NLLanguage.english print("Current revision:", NLEmbedding.currentSentenceEmbeddingRevision(for: language)) print("Supported revisions:", Array(NLEmbedding.supportedSentenceEmbeddingRevisions(for: language))) if let model = NLEmbedding.sentenceEmbedding(for: language) { print("Loaded revision:", model.revision, "dimension:", model.dimension) } else { print("Sentence embedding model unavailable") } On affected devices, the current revision is 1 and supported revisions are [1], but model loading returns nil. The framework logs: Unable to locate Asset for sentence embedding model for local en. The failure occurs during model loading, before any call to vector(for:). This uses Apple’s built-in English sentence model, with no custom embedding file. Is this a known issue on iOS 27? Is there a supported way to request or restore the missing sentence-model asset, or a recommended recovery approach?
0
0
480
1w
Vision face landmarks shifted on iOS 26 but correct on iOS 18 with same code and image
I'm using Vision framework (DetectFaceLandmarksRequest) with the same code and the same test image to detect face landmarks. On iOS 18 everything works as expected: detected face landmarks align with the face correctly. But when I run the same code on devices with iOS 26, the landmark coordinates are outside the [0,1] range, which indicates they are out of face bounds. Fun fact: the old VNDetectFaceLandmarksRequest API works very well without encountering this issue How I get face landmarks: private let faceRectangleRequest = DetectFaceRectanglesRequest(.revision3) private var faceLandmarksRequest = DetectFaceLandmarksRequest(.revision3) func detectFaces(in ciImage: CIImage) async throws -> FaceTrackingResult { let faces = try await faceRectangleRequest.perform(on: ciImage) faceLandmarksRequest.inputFaceObservations = faces let landmarksResults = try await faceLandmarksRequest.perform(on: ciImage) ... } How I show face landmarks in SwiftUI View: private func convert( point: NormalizedPoint, faceBoundingBox: NormalizedRect, imageSize: CGSize ) -> CGPoint { let point = point.toImageCoordinates( from: faceBoundingBox, imageSize: imageSize, origin: .upperLeft ) return point } At the same time, it works as expected and gives me the correct results: region is FaceObservation.Landmarks2D.Region let points: [CGPoint] = region.pointsInImageCoordinates( imageSize, origin: .upperLeft ) After that, I found that the landmarks are normalized relative to the unalignedBoundingBox. However, I can’t access it in code. Still, using these values for the bounding box works correctly. Things I've already tried: Same image input Tested multiple devices on iOS 26.2 -> always wrong. Tested multiple devices on iOS 18.7.1 -> always correct. Environment: macOS 26.2 Xcode 26.2 (17C52) Real devices, not simulator Face Landmarks iOS 18 Face Landmarks iOS 26
Replies
1
Boosts
0
Views
559
Activity
5h
Codex GPT-5.5 got deprecated, no other models selectable in xcode
Oldest model of 5.5 with codex is deprecated by chatgpt but I only see 5.5 as the highest available model selectable in the dropdown in intelligence settings in Xcode when selecting codex I have the ChatGPT pro account and login to codex in xcode via email/password and not the API key I have already tried login out, quitting xcode, removing codex agent and reinstalling it, so far nothing works
Replies
3
Boosts
0
Views
638
Activity
10h
iPhone 18 Pro: First Core ML load on Neural Engine up to 65× slower than on iPhone 17 Pro
On iPhone 18 Pro (A20 Pro), the first MLModel load of a model on the Neural Engine can take minutes where an iPhone 17 Pro needs seconds. Cached loads are instant, and once loaded the model runs faster than on the iPhone 17 Pro, so only the on-device specialization is affected. This looks like the same problem as A20 pro devices take too long on CoreML model specialization and iPhone 18 pro takes very long time to load models. The "loading" time there is likely this specialization too: it happens inside MLModel(contentsOf:) for an already compiled .mlmodelc. I can reproduce it reliably with attention in the form Apple recommends for the Neural Engine (split einsum, as in Deploying Transformers on the Apple Neural Engine). In this example most of the extra time comes from the softmax over the key axis, which in that layout is the channel axis. Other ops and model types may well be affected too; this is simply the smallest case we could build that shows the problem clearly. The repro uses four self-attention layers (10 heads × 64, 4096 tokens, fp16) built with coremltools and loaded cold with .cpuAndNeuralEngine. MLComputePlan places every op on the Neural Engine on all three devices. iOS 27.0.1 (24A446) on both iPhones: iPhone 18 Pro iPhone 17 Pro M3 Pro Mac cold predict cold predict cold predict Split-einsum attention, 4 layers 53.4 s 21.6 ms 0.8 s 111.4 ms 1.0 s 105.8 ms Q.K einsum only 5.1 s 24.0 ms 1.2 s 36.8 ms 1.4 s 41.8 ms Q.K einsum + softmax 36.1 s 61.5 ms 1.7 s 141.0 ms 2.0 s 128.3 ms Scores keys-last (softmax last axis) 10.5 s 109.0 ms 0.6 s 198.4 ms 0.7 s 180.2 ms scaled_dot_product_attention 1.4 s 140.4 ms 0.4 s 184.9 ms 0.5 s 148.1 ms Cached loads take 0.01–0.03 s everywhere. What I observed: Compile time grows linearly with the number of attention layers. Every alternative attention layout I tried that compiles quickly runs 5–6× slower on the A20 Pro, so I found no workaround that keeps its speed. During the slow load, ANECompilerService runs almost entirely on the efficiency cores. Requesting a higher QoS, MLOptimizationHints (.fastPrediction, reshapeFrequency), computeUnits = .all and dense instead of palettized weights don't help. Loading several models in parallel additionally gets ANECompilerService killed by jetsam (per-process-limit). The specialization cache doesn't survive reinstalling the app. In our app (Stable Diffusion XL), each UNet chunk with attention takes 110–150 s instead of 6–16 s, so the first launch takes about 25 minutes in the foreground, and in our tests again after every reinstall. The same networks converted to Core AI specialize normally on this device (see our Core AI porting notes). Filed as FB25106520 with a self-contained repro: a coremltools script, an iOS app and a Mac runner, plus results from all three devices. Environment: iPhone 18 Pro (iPhone19,2) and iPhone 17 Pro (iPhone18,1) on iOS 27.0.1 (24A446), M3 Pro on macOS 27.0.1 (26A434), Xcode 27.0, coremltools 8.3.0. Is this a known A20 Pro issue, and is there a recommended way to avoid it until it's fixed? Falling back to the GPU or CPU isn't a good option for us: it avoids the slow load, but gives up the Neural Engine's efficiency, which matters for on-device generation on a phone, and its speed (a full SDXL UNet step on the iPhone 18 Pro takes 509 ms on the Neural Engine vs 1,639 ms with .cpuAndGPU). We'd also like to hear from others which ops their slow first loads involve, to see whether this is specific to attention and softmax or broader.
Replies
8
Boosts
5
Views
1.5k
Activity
1d
What signal should drive fallback for PrivateCloudComputeLanguageModel?
I'm building an app that uses PrivateCloudComputeLanguageModel as the primary inference tier with SystemLanguageModel as the fallback. The app is entitled (com.apple.developer.private-cloud-compute, granted and provisioned) and generations serve normally. My question is how a client should decide to fall back because in extended measurement, no public signal ever reflects the blocked state I actually hit. What I measured (macOS 27.0 beta, 26A5416b / Xcode 27 beta 27A5237l, entitled signed bundle constructing PrivateCloudComputeLanguageModel directly): Serving stopped mid-run with no leading signal: request N served normally (1.4 s), request N+1 threw LanguageModelError.rateLimited 494 ms later, at cumulative generation 786 for the day. 100% served → 100% refused between consecutive calls. Every quota signal read healthy the entire time: before, during, and after the block. Across 1,517 readings in a single day: quotaUsage.status = belowLimit, isApproachingLimit = false, isLimitReached = false, resetDate = nil, availability = .available. A preflight on these APIs cannot see the condition. The refusal is enforced locally after first contact: rejections return in ~230 ms vs ~0.9–1.4 s for served calls, so the client appears to cache the verdict rather than ask the server per-request. The trigger is a cumulative ledger, not a request rate: 501 generations at 33/min in one 15-minute sitting was fine, and a later arm sustained 39.7/min; two bursts of 16 concurrent at 5.0 and 5.2 req/s served 32/32; the count that tripped survived a process restart and a 4.9-hour idle gap. But it's not a fixed daily number either. 501 fast was fine earlier the same day; the trip came 285 requests later. A rolling window on the order of hours-to-a-day is consistent with this, but nothing here measures its length. Recovery: still blocked at +41 minutes (probes at +1/2/5/10/20/40 min all refused); fully recovered by +20 h with no intervention and no upgrade. Next day served normally from the first request. quotaLimitReached never occurred: not once in ~800 generations plus the blocked period. The wall is typed as the transient error while carrying what the documentation describes as daily quota semantics ("a person either waits for their usage quota to refresh or they upgrade"). limitIncreaseSuggestion is presence-constant: nil at process start, non-nil on every reading after first PCC contact (identical while fully serving and while fully blocked) so its presence can't gate an upsell affordance. The same signals-read-healthy-while-refusing divergence also reproduces against the developer-tool pool (fm serve), which I've reported separately (FB24273854 covers quota exhaustion surfacing there as a generic server_error/500 while /health reports the model available). Questions: Is attempt-and-classify the intended contract? Given that no preflight can observe the blocked state, should a client simply issue the request, treat the typed error as authoritative, and route to SystemLanguageModel? And is the ~230 ms local fail-fast on the blocked path contractual (cheap and safe to probe) or incidental? This is the one that decides how I ship; the rest are diagnostics behind it. What does quotaUsage actually track, and at what granularity? I have driven the entitled app-tier path to a hard block and the developer-tool pool to exhaustion, and no field ever moved. Is there any consumption pattern that moves isApproachingLimit / isLimitReached / resetDate? If the intended answer is "only the per-person daily quota, which these volumes never approached," what is the wall I am hitting at ~786 cumulative, and why does it surface as rateLimited? Should rateLimited and quotaLimitReached drive different client behavior — and which one is the daily allowance in practice? The documentation distinguishes rate limiting ("wait a period and retry") from daily exhaustion ("wait for refresh or upgrade"), but what I observe is the transient-typed error carrying the multi-hour ledger semantics. Concretely: what retry cadence is recommended after rateLimited (my measured recovery horizon was somewhere between 41 minutes and 20 hours. My current design stays on the on-device model and re-probes PCC at a low fixed interval rather than per-request)? And under what condition is resetDate ever populated, given it was nil even while blocked? (Smaller, design guidance): my app can generate a few hundred requests as one feature batch (quiz generation over a user's imported document). Measured: 501 in a sitting was fine, cumulative 786 in a day was not. Since this allowance belongs to the person and is shared with every Apple Intelligence feature, is a several-hundred-request batch a reasonable use of it, or should features like this generate on demand? (I'm aware of the existing feature request for richer quota reporting (FB23378161); this is a narrower design question.) I can attach the measurement driver and timestamped JSONL logs. The divergence is reproducible on a fresh day, though reaching the wall took ~800 cumulative generations.
Replies
5
Boosts
1
Views
2.1k
Activity
2d
iOS 27 + RecognizedText misses lines
I'm using the iOS 18+ RecognizedTextRequest, and with no change of code, iOS 27 gives poorer results: lines are dropped, all the text is not recognized it's not my code, because Live Text has the same issue and of course, back to 26 or 18 solves the issue What changed? Only interesting infos I could isolate: The simulator has not the issue Squashing the image horizontally (80% of its width for example) makes the detection significantly better
Replies
6
Boosts
5
Views
1.8k
Activity
3d
Major iOS 27 Regression in '.accurate' Model for VNRecognizeTextRequest
Hello, With the update from iOS 26 to iOS 27 I have noticed significant regressions and degraded performance when using the Vision Framework for OCR via 'VNRecognizeTextRequest'. The degraded performance is only applicable to the .accurate model, the .fast model is identical between iOS 26 and iOS 27. I have an app that uses this framework extensively for extracting information from the video feed of professional cinema cameras. This runs using en-US, language correction off with a specific region of interest. On iOS 26 everything works reliably as it has for the past few years, with the exact same codebase on iOS 27 many strings that were previously detected are no longer detected, some strings return as if they've been read upside down (ISO1280 is returned as 08ZLOSI) and there are a lot of Cyrillic look alikes mixed into Latin text, there also appears to be a lot more junk characters tacked onto the end of numbers ('12ł' is returned when previously it was just '12'). You often see whole line duplicates too where the same text is recognised twice, once as a single line with all strings combined and once as individual strings. The iOS 27 Simulator doesn't show any of this. I only saw it on real iPads, if you only test in the Simulator you'll miss it. To put numbers on it, I built a test set of about 26,500 frame grabs and ran them through 'VNRecognizeTextRequest' on iPadOS 18, 26 and 27. With the version of my app that's on the App Store today, iOS 26 reads 99.7% of values correctly. On iOS 27 that drops to 65%. I've resolved this issue now and have been able to retain the level of performance that existed on iOS 26, this involves various preprocessing steps to the images as well as a significant amount of validation after the Vision Framework returns its results. These fixes took around 3.5 Days of work which is incredibly frustrating. I would much rather use my time building new features instead of fixing regressions introduced by poorly validated new models that have been shipped in iOS 27. What frustrates me most is this change wasn't announced anywhere and there's no way to opt out. The revision number didn't change so there's no way to keep using the model that worked. If this had shipped as VNRecognizeTextRequestRevision4 apps could have stayed on VNRecognizeTextRequestRevision3 until they'd had a chance to test. Or permanently stayed on 'VNRecognizeTextRequestRevision3' as obviously the new version performs significantly worse. Instead an app that worked fine on iOS 26 now gets wrong text back on iOS 27 with no error, no warning and nothing in the release notes. For a framework people rely on for real work that's a significant regression. It seems that other developers have also come across this issue: https://developer.apple.com/forums/thread/845452 For technical reference, if you diff System/Library/PrivateFrameworks/TextRecognition.framework between iOS 26.5 (23F77) and iOS 27.0 (24A434) you uncover a number of changes. New models on iOS 27 seem to use an '_e5' variant. The text detector in iOS 26 references training run 'yyzn76u6fs' with 88560 iterations, iOS 27 references training run 'hvcthxumvz' with only 10116 iterations. Perhaps an 8x reduction in iterations has something to do with the poor performance in iOS 27? Further, the orientation model changes from iOS 26 'cr_orientation_model_v1' with 1858472 iterations vs. iOS 27 'cr_orientation_model_v1_e5' with 671065 iterations. The Latin and Cyrillic Recogniser changed model entirely from 'cr_tr_model_latincyrillic_v3' on iOS 26 to 'cr_tr_model_latincyrillic_v3_e5' on iOS 27. It seems these 'e5' models are optimised for the neural engine. Is there any way to get the iOS 26 models/behaviour back or plans to back deploy the old model in iOS 27.1 or iOS 27.2? Can future model changes like this come with a new revision number? What testing process does the Vision Team use to validate models as clearly something was missed here? Thanks!
Replies
0
Boosts
0
Views
260
Activity
3d
iOS 27: ML Program pipeline with data-dependent shapes (NMS / gather / non_zero) fails on every compute unit and allocates ~2.8 GB at load
A Core ML model that we used in production on iOS 26 no longer works on iOS 27.0.1. Prediction now fails on every compute unit. Loading it with .all also increases the process footprint by about 2.8 GB, so on 6 GB devices jetsam kills the app shortly after launch (per-process-limit). Environment iPhone 13 Pro (iPhone14,2), iOS 27.0.1 (24A446): the memory spike and the prediction failure both reproduce. The app is killed about 1.5 s after launch. iPhone 15 Pro, iOS 27.0.1 (24A446): the memory spike reproduces (about 3 GB). The app is not killed because this device's limit is higher. macOS 27.0.1 host, small Swift CLI using Core ML: the prediction failure reproduces. The same model and app build worked on iOS 26.x. On an iPhone 15 Pro running iOS 26.6.1, the app peaked at about 570 MiB. Model A compiled .mlmodelc (from an .mlpackage). It is a pipeline of two ML Programs. The input is a fixed-size 640×480 image. model 0: a YOLO-style instance segmentation network converted from TensorFlow. It includes non_maximum_suppression over 1500 candidate boxes. The NMS is followed by range_1d → gather with data-dependent [?] shapes. There is a zero-sized intermediate tensor (tensor<fp16, [1500, 0]>). Results are written with scatter_nd into a fixed-size output [1, 300, 38], plus prototype masks [1, 120, 160, 32]. model 1: post-processing converted from PyTorch. It runs non_zero over the 300 candidates. After that it works on dynamic [?, 1, 480, 640] tensors (upsample_bilinear, greater, cast, mul), then uses tile and reduce_argmax. Symptom 1: prediction fails on all compute units The failure is the same with .all, .cpuAndNeuralEngine, .cpuAndGPU and .cpuOnly, with a real camera image as input: Error Domain=com.apple.CoreML Code=0 "Failed to evaluate model 0 in pipeline" -> "Unable to compute the asynchronous prediction using ML Program. It can be an invalid input data or broken/unsupported model." -> "E5RT: Submit Async failed for [6:1]: Unable to successfully execute plan. @ ExecuteSync (11)" On macOS 27, the following is logged when the model loads: MIL program has non-constant (dynamic) shapes for external input but FlexibleShapeInformation attribute is missing. E5 function will be produced with unknown input shapes and cannot be run without reshaping first. Got input with flexible dimension but no default shapes provided. Will use 1 Symptom 2: about 2.8 GB allocated at load This is phys_footprint right after MLModel(contentsOf:configuration:). The model was loaded alone in a fresh process on an iPhone 13 Pro running iOS 27.0.1. The baseline before loading was 13 MiB. .all: 2858 MiB .cpuAndNeuralEngine: 2857 MiB .cpuAndGPU: 2515 MiB .cpuOnly: 62 MiB The other models in the same app stay under about 520 MiB on iOS 27 (load plus one prediction each). They include object detection, pose estimation and a ViT that runs on the ANE. Another model in the app with data-dependent shapes, a keypoint detector, still loads and predicts correctly on iOS 27, so not every dynamic-shape model is affected. In the field, the JetsamEvent report shows per-process-limit at 3072 MiB. A diskwrites_resource report shows the heaviest stack under Core ML model load → Espresso → MetalPerformanceShadersGraph. Current workaround Loading this model with .cpuOnly on iOS 27 avoids the memory spike, so the app no longer crashes. Prediction still fails, though, so the model can't be used on iOS 27. Questions Is this a known iOS 27 regression in how E5RT handles data-dependent shapes inside ML Program pipelines (NMS, gather, non_zero, zero-sized tensors)? What is the recommended way to export a model like this for iOS 27? Would adding flexible-shape information help, or should NMS and mask post-processing be moved out of the model? Is it expected that unbounded dynamic shapes cause a worst-case allocation of several GB at load time with GPU or ANE compute units? I can't share the model itself, but I'm happy to provide more details (full MIL op listing, logs, sysdiagnose) through Feedback Assistant if that helps.
Replies
2
Boosts
0
Views
412
Activity
3d
iPhone 18 pro takes very long time to load models
I think this could be relate to previous post here https://developer.apple.com/forums/thread/848590 But in my testing it seems like most of the time spent was in loading but not compilation. Our app has multiple models but so far it seems like one of them has this issue. The model itself was only 2.2MB, but somehow loading took 50s after compilation? And on 17e it only took 2s to load. But I cannot attach the model here so please advice.
Replies
2
Boosts
2
Views
1k
Activity
4d
About the IOS 27
i have a question about the iOS 27, is there an exact date yet for iOS 27 with the new features like Siri AI and everything else, is there a date when iOS 27 comes out in Europe with the new Siri AI?
Replies
0
Boosts
0
Views
181
Activity
4d
Core ML memory usage is dramatically higher with an Xcode 27 build on iOS 27
I’m seeing a major change in reported memory usage (and eventual termination due to memory pressure) when running a Core ML workload built with Xcode 27 on iOS/iPadOS 27. The source code, model files, and MLModelConfiguration are unchanged. Only the Xcode/SDK version used to build the app differs. On the same iPad running iPadOS 27: Xcode 26 build: model loading and prediction complete normally, with a relatively small reported application footprint. Xcode 27 build: the application footprint grows continuously as models are loaded and can exceed 5 GB. The app is eventually terminated unless models are unloaded very aggressively or the increased-memory-limit entitlement is used. I also tested an Xcode 27 build on a device running iOS 26. Its reported peak was only around 300 MB. This suggests the change requires both an Xcode 27-linked binary and the iOS 27 runtime. The workload consists of several compiled Core ML models using .cpuAndNeuralEngine. Loading models sequentially instead of concurrently does not materially change the final footprint. Releasing each MLModel after use does reduce it, so this appears to be model or Neural Engine residency being charged to the application rather than a conventional heap leak. I noticed that the iOS 27 release notes mention Neural Engine memory now being attributed to the application instead of the system. However, I’m unclear about the practical consequences of that change. If the same Neural Engine resources were already physically resident on iOS 26, I would have expected them to contribute to system memory pressure even when they were not attributed directly to the application. Instead, the older configuration runs comfortably, while the Xcode 27/iOS 27 combination approaches or crosses the application’s per-process memory limit. A few additional observations: The problem is more likely to occur after Core ML has already compiled and specialized the models. Cached model loading is much faster and the footprint grows quickly. The first uncached run can survive model preparation because specialization spaces the loads farther apart. Under Instruments, the app often does not terminate, presumably because profiling slows the workload enough to change the peak. os_proc_available_memory() decreases in line with the newly reported footprint. With the increased-memory-limit entitlement, the workload completes, but the reported footprint still reaches several gigabytes. Has anyone else observed a large Core ML memory increase specifically with an Xcode 27 build running on iOS 27? In particular, I’m trying to understand: Is this purely a change in how existing Neural Engine memory is accounted for, or does the new runtime also retain or allocate more memory? Is the new accounting used for the application’s jetsam/per-process memory limit? Is this behavior intentionally gated by the linked SDK version? That would explain why an Xcode 26 build behaves differently on the same iOS 27 device. Should applications now treat the Neural Engine residency of every loaded MLModel as part of their process-memory budget and unload models accordingly? Are there recommended APIs or Core ML loading strategies for controlling this residency? Any confirmation that others are seeing the same Xcode 27/iOS 27 behavior—or clarification of the intended memory-accounting model—would be very helpful.
Replies
2
Boosts
3
Views
1.4k
Activity
4d
A20 pro devices take too long on CoreML model specialization
Hi, I'm a ML developer specializing in local audio related models. I upgraded from a 16 pro max to an 18 pro max. I'm not sure what the reason is, but model specialization (happens only on the first load, cpu and neural engine compute units) is sometimes 2x - 5x longer on the 18 pro max vs the 16 pro max. Both devices are on iOS 27. For example, I have a stem separation model running, and the 18 pro max took 171 seconds (including inference) and the 16 pro max took 37 seconds (including inference). 18 pro max inference is ~55x RTFx and the 16 pro max is ~40 RTFx for this particular model, but I've noticed it as well in TTS models etc. Please look into this, I know the 18 pro max has double the neural engine cores, so I'm not sure if the complexity makes specialization take longer. Subsequent loads are great, same high rtfx for the 18 PM (only 0.5s load), and this is where it beats the 16 PM handily. I attached 2 images showing the 18 PM on a first load (model specialization likely) vs a subsequent run. Please look into this. (Note this is not just my implementation, I tested a range of different models, all with the same outcome).
Replies
5
Boosts
6
Views
1.6k
Activity
4d
Adding MCP and connector support to your own Foundation Models apps
Circling back on the LocalLM Lab arc. With v0.7, we've moved from prompt experimentation into real app development on Apple's Foundation Models local AI. The LocalLM Lab SDK lets you build that same on-device model and MCP client this thread has covered directly into your own app, with real tool and data access (Slack, Todoist, GitHub, Notion, Linear, plus Calendar, Reminders, Contacts and Location). And you can ship your app including through the Mac App Store. This is a big improvement over version 0.6, where the localai-cli toolkit needed LocalLM Lab installed and running. On the other hand, the SDK (LocalLMLabSDKCore) doesn't relay through anything; it links FoundationModels and a real MCP client directly into your own binary and is totally self-contained. The example included in the SDK, Plate Today, has actually been built into a sandboxed test app and verified working, with a signed path to a Mac App Store .pkg (Apple Distribution signing + provisioning profile pipeline). That's "verified signable and sandbox-compatible," to be precise. Entitlements (from personal experience: always a complicated topic): com.apple.security.app-sandbox + com.apple.security.network.client for the app itself, plus the standard personal-information entitlements per connector used (com.apple.security.personal-information.calendars, .addressbook, .location) and matching NS*UsageDescription strings in Info.plist. The one worth flagging specifically: the network entitlement is easy to miss and fails silently rather than throwing. Without it, MCP connections and Weather calls just hang with no error surfaced. OAuth handling requires the app delegate callback (application(_:open:)), not SwiftUI's .onOpenURL. Worth knowing before wiring it up if you're SwiftUI-only. Full entitlements list + SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/sdk.html I hope the availability of the SDK (free, Apache 2.0 license) will give folks further incentive to explore local AI-enabled applications on the Mac. What else would you want to do that the SDK doesn't currently support? File picker? Calendar/Reminders/Contacts edits & writes?
Replies
7
Boosts
1
Views
4.2k
Activity
6d
Model Guardrails Too Restrictive?
I'm experimenting with using the Foundation Models framework to do news summarization in an RSS app but I'm finding that a lot of articles are getting kicked back with a vague message about guardrails. This seems really common with political news but we're talking mainstream stuff, i.e. Politico, etc. If the models are this restrictive, this will be tough to use. Is this intended? FB17904424
Replies
11
Boosts
5
Views
3.1k
Activity
6d
Framing an object in real time
Hi, I am currently working on a project which involves giving directions to the user telling them what they should do to frame a given object (also only doing this if sees the object which can be many things, so it needs detection too) so that it can be optimally in view of the camera to be classified. I have figured the classification, but not the real time guidance and detection. I tried training a model that featured all of the different movements ("move up", "move down", etc.). It featured 30 different objects with ten crops per image, and each class featured all 30 images and an even amount per class. It had over 1500 images in total, and I trained using createml and core ml. But, they were very inaccurate in my testing. I also tried using foundation models for this, but it also didn't work. I am quite new to integrating ai/ ml models into a project, and I am unsure of where to go next. If anyone has any suggestions, I would be really appreciative.
Replies
0
Boosts
0
Views
332
Activity
1w
Personal Voice UPGRADE
I have created a Personal Voice and have selected it in my AI app. It sounds awful. Why? Can I continue to train it until its sounds more like me? Why not? When should I expect a Personal Voice Upgrade.
Replies
0
Boosts
0
Views
518
Activity
1w
SiriAI Voice my app
Policy or no policy I think its clear if Apple wants to have a premium SiriAI experience its gonna need to allow apps to use the Siri AI Voice engine Developers like me want the privacy/no cloud voice engine service that a SiriAI Voice would provide Seemless conversations with versions of SiriAI Voices let's users know they're still in the SiriAI ecosystem but engaged with a developer app The "hey Siri, ask LinkedIN Hoffman to .." is stunningly haphazard moat gate that Apple can resolve and still keep control with a simple App License so it know every app that uses a SiriAI Voice App engine. I've been waiting over a year for Apple to figure this out. This is an unforced error. This is not a threat its an opportunity for Apple ? Is this a Driver or CoreOS issue? WTF?
Replies
0
Boosts
0
Views
339
Activity
1w
Golden Gate - 4 weeks after install "adding support in progress"
I have tech support case running -- no help at all after over 2 weeks these files explain what I have tried, to no avail Siri AI Findings.txt Siri_AI_Diagnostic_FOR_APPLE_FORUM.txt Thoughts?
Replies
0
Boosts
0
Views
456
Activity
1w
Dutch Siri identifies my App Shortcut but requests Notes or falls back to Home/Intercom — FB25058524
I need a supported way for Dutch Siri to invoke Elegant Cook's built-in grocery actions directly, without requiring users to create personal shortcuts. The installed TestFlight version is 1.5.3 (32), on an iPad Air (5th generation). The owner reports iPadOS 26.6.1; the diagnostic archive identifies build 23G83. Siri is Dutch, with nl_BE runtime assets. Use with Siri is enabled and spoken responses are preferred. The main app uses App Intents and AppShortcutsProvider with three actions: add a grocery, read the list, and check off a grocery. The voice actions use scalar String parameters with requestValueDialog and return ProvidesDialog results. The phrases use the applicationName token and have packaged English and Dutch localizations. We do not register Notes or Home intents. Observed behavior: “Hé Siri, voeg boodschappen toe in Elegant Cook” and “Hé Siri, lees mijn boodschappenlijst in Elegant Cook” ask whether to use Elegant Cook or Notes. After selecting Elegant Cook, adding and reading work by voice. “Hé Siri, vink een boodschap af in Elegant Cook” and the registered app-name-first equivalent “Hé Siri, Elegant Cook boodschappen afvinken” instead produce the instruction to download Apple's Home app to use Intercom. “Hé Siri, open Elegant Cook” opens the installed app correctly. The signed build contains extracted App Intents metadata, compiled Dutch/English nlu.appintents assets and successful phrase-training output. These build checks do not establish successful device routing or App Shortcuts Preview validation. For one captured check-off invocation, local diagnostic analysis identified this sequence: the recognized utterance includes the correct app name; App Shortcuts are enabled; the intended CheckElegantCookGrocery action is a Siri candidate; app disambiguation then fails and explicitly redirects to Home/Intercom before the grocery operation. This trace covers check-off only. It does not prove that the add/read chooser has the same cause, or establish a definitive Apple framework bug. The private bug report is FB25058524. It includes selected evidence and source code. A full sysdiagnose has not been uploaded, and the isolated Swift example is not a compiled or validated reproducing Xcode project. We develop on Windows and use EAS cloud builds, so local Xcode App Shortcuts Preview is unavailable. The code-level support form currently directs us to this forum when we cannot provide a test project demonstrating the issue. Could an App Intents/DTS engineer advise: Which supported registration or metadata correction should we check when Siri identifies an explicitly named App Shortcut but chooses another domain before perform()? What targeted diagnostic evidence can distinguish an app configuration problem from a Siri routing defect without local App Shortcuts Preview? Is there a supported workaround for direct Dutch voice invocation on this OS build, without personal shortcuts or private system settings? We can prepare an isolated native sample project if required, but will clearly distinguish source preparation, cloud compilation and actual device reproduction. We do not want to claim a reproduction that has not been verified.
Replies
0
Boosts
0
Views
421
Activity
1w
iOS 27: concurrent Core ML predictions on the Neural Engine are serialized
Environment iPhone 17 Pro Max, iOS 27.0.1 ([build]) Reference: MacBook Pro (M3 Pro), macOS 26.4.1 ML Program model, MLComputeUnitsCPUAndNeuralEngine. A single prediction takes about 2.8 ms on the ANE. Description On iOS 27, Core ML predictions submitted to the Neural Engine from multiple threads at the same time no longer overlap. The ANE runs them strictly one after another. Adding threads gives no extra throughput, and per-call latency grows linearly with the number of requests in flight. The same code still gets a clear concurrency gain on macOS 26.4.1, and our app's ANE-parallel path on this iPhone became noticeably slower after the iOS 27 update. On iOS 27, latency is about N × single-call time, which means requests are queued and never overlap. Running two different models concurrently on the ANE shows the same behavior. What we tried, with no change in result A separate MLModel instance per thread. The async predictionFromFeatures:options:completionHandler: API. Questions Is this serialization an intended change in iOS 27? Is there a supported way to get concurrent ANE execution back?
Replies
0
Boosts
0
Views
462
Activity
1w
iOS 27: English sentence embedding model unavailable on physical devices
I’m seeing NLEmbedding.sentenceEmbedding(for: .english) return nil on physical devices running iOS 27, both in debug builds under Xcode and in production. The same failure has been observed across thousands of users on iOS 27. One affected device runs iOS 27.0.1 (24A446). Minimal example with diagnostics: import NaturalLanguage let language = NLLanguage.english print("Current revision:", NLEmbedding.currentSentenceEmbeddingRevision(for: language)) print("Supported revisions:", Array(NLEmbedding.supportedSentenceEmbeddingRevisions(for: language))) if let model = NLEmbedding.sentenceEmbedding(for: language) { print("Loaded revision:", model.revision, "dimension:", model.dimension) } else { print("Sentence embedding model unavailable") } On affected devices, the current revision is 1 and supported revisions are [1], but model loading returns nil. The framework logs: Unable to locate Asset for sentence embedding model for local en. The failure occurs during model loading, before any call to vector(for:). This uses Apple’s built-in English sentence model, with no custom embedding file. Is this a known issue on iOS 27? Is there a supported way to request or restore the missing sentence-model asset, or a recommended recovery approach?
Replies
0
Boosts
0
Views
480
Activity
1w